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FOREWORD 


This  manuscript  is  the  final  report  of  an  investigation  carried  out 
for  the  Air  Force  under  Contract  AF41(657)274  with  the  University  of 
Texas.  The  principal  investigator  was  Dr.  Benjamin  Fruchter,  Department 
of  Educational  Psychology,  University  of  Texas.  The  contract  was  moni¬ 
tored  for  Personnel  Research  Laboratory  by  Major  Joseph  M.  Madden. 

On  the  University  of  Texas  staff.  Dr.  Robert  E.  Morin  served  as  Re¬ 
search  Scientist  and  Mr.  Jack  E.  Capehart  as  Social  Science  Research 
Associate.  Mrs.  Abigail  B.  Capaldi  and  Mr.  Wayne  B.  Archer  were  Social 
Science  Research  Assistants. 


ABSTRA  'T 

Checklists  of  tasks  included  in  an  Air  Force  specialty  are  used  to  collect  job  informa¬ 
tion  from  incumbents,  with  provision  for  them  to  write  in  tasks  they  perform  which  are  not 
listed.  This  study  investigated  methods  of  selecting  incumbents  and  presenting  the  checklist 
to  produce  the  most  complete  and  accurate  task  inventory.  Incumbents  of  4  AFSCs  (Ground 
Radio  Operator,  Automotive  Repairman,  Aircraft  Hydraulic  Repairman,  Accounting  &  Finance 
Specialist)  were  selected  to  be  representative  of  commands  and  geographic  location.  Portions 
of  the  samples  were  given  inventory  forms  that  intentionally  omitted  some  tasks  known  to  be 
part  of  the  job.  From  a  tally  of  write-ins,  rate  of  retrieval  of  omitted  tasks  and  expected  pro¬ 
duction  of  new  task  statements  were  computed  for  3  sample  sizes  (20,  40,  60)  within  each 
AFSC.  By  extrapolating  curves  fitted  to  the  data,  it  was  estimated  that  samples  of  100  incum¬ 
bents  would  yield  85%  of  the  task  statements  produced  by  the  full  sample  (360).  About  25% 
wrote  in  no  additional  tasks,  50%  no  more  than  3,  and  only  rare  individuals  over  20.  Multiple 
regression  analyses  revealed  no  effective  combination  of  predictors  to  identify  productive 
individuals.  Aircraft  Hydraulic  Repairmen  produced  the  least,  Accounting  &  Finance  Spe¬ 
cialists  the  most  new  statements.  Expanded  task  inventories  were  completed  by  a  second 
sampling  of  incumbents  who  rated  each  task  they  performed  for  time  required,  frequency  of 
performance,  and  training  &  experience  required.  Another  series  of  multiple  regression 
analyses  showed  that  only  the  number  who  reported  performing  a  task  was  highly  related  to 
likelihood  of  a  task  being  written  in. 


This  report  has  been  reviewed  and  is  approved. 


Fred  E.  Holdrege,  Col,  USAF  A.  Carp 

Commander  Technical  Director 

Hq  6570th  Personnel  Research  Laboratory 
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EFFICIENCY  OF  THE  OPEN-ENDED  INVENTORY  IN  ELICITING 
TASK  STATEMENTS  FROM  JOB  INCUMBENTS 


I.  INTRODUCTION 

This  study  is  part  of  a  broad  program  of  research  in  the  area  of  occupational  analysis 
and  description  sponsored  by  the  Air  Force.  Morsh  .*  Ratliff  (1959)  report  that  occupational 
analysis  "forms  part  of  a  continuing  progtam  designed  to  improve  selection  and  training 
standards;  to  develop  efficient  and  flexible  assignment  procedures;  to  provide  guide  lines 
for  estimation  and  planning  of  future  manpowet  requirements;  to  establish  bases  for  equable 
job  evaluation;  and  to  provide  criteria  for  promotion  or  reassignment  of  personnel”  (p.  1). 

Job  analysis,  an  important  component  of  occupational  analysis,  is  concerned  with  the 
collection,  processing,  and  interpretation  of  information  relevant  to  the  work  men  do  on  their 
jobs.  Until  recently  the  principal  job  analysis  procedures  employed  by  the  Air  Force  were 
group  interviews  of  job  incumbents  and  conferences  of  technical  experts.  Morsh,  Madden,  and 
Christal  (1961)  describe  newly  revised  procedures  which  are  expected  to  improve  the  quality 
of  job  information  and  the  efficiency  of  its  collection. 

Under  the  new  procedures  a  task  inventory  is  the  principal  instrument.  In  essence,  a 
task  inventory  is  a  list  of  statements  describing  the  tasks  of  a  particular  job.  Incumbents 
provide  information  by  checking  tasks  they  have  performed,  during  a  specified  period  of  time, 
which  are  part  of  their  regular  job.  For  those  tasks  which  are  checked,  the  incumbent  may 
further  indicate  how  frequently  he  has  performed  the  task,  how  long  it  usually  takes,  how 
difficult  it  is,  etc.  Furthermore,  he  is  instructed  to  write  in  any  tasks  which  he  performs 
which  are  not  included  in  the  existing  list. 

Morsh  el  al.  (1961)  consider  the  merits  of  the  task  inventory  method  in  some  detail. 

One  of  the  most  important  advantages  cited  is  that  the  method  allows  for  large  samples  of 
incumbents  to  be  used  on  a  relatively  unrestricted  basis.  Broad  sampling  is  possible  because 
inventories  are  essentially  self-administering.  They  cf.n  be  completed  by  incumbents  in  test¬ 
ing  rooms  or  at  their  worksites. 

In  an  organization  as  large  as  the  Air  Force,  with  members  of  any  one  specially  stationed 
at  bases  throughout  the  world,  the  feasibility  of  relatively  unrestricted  and  respresentative 
sampling  is  especially  important.  It  is  well  known  that  the  tasks  performed  by  men  within  a 
given  specialty  may  vary  markedly  as  a  function  of  climate,  unit  mission,  base  facilities,  type 
of  air  craft  or  missile,  size  of  military  unit,  etc.  Restricted  sampling  may  yield  a  biased  pic¬ 
ture  of  the  tasks  performed  within  a  specialty. 

Another  advantage  of  the  task  inventory  method,  one  related  to  the  first,  is  its  sensitivity 
to  changes  in  the  character  of  jobs.  The  Air  Force  is  a  dynamic  organization  in  the  forefront 
of  technological  changes  in  our  society.  Jobs  change  as  technology  advances.  However, 
changes  do  not  occur  at  the  same  rate  at  all  bases  and  for  all  specialists.  If  information  is 
collected  directly  from  men  who  perform  a  job,  and  if  an  adequate  sample  of  men  is  selected, 
it  should  be  possible  to  update  job  information  almost  as  rapidly  as  changes  occur. 

Purpose  of  the  Present  Research 

Problems  of  constructing  task  inventories  and  problems  associated  with  the  use  of 
completed  instruments  define  two  somewhat  distinct  areas  of  research.  Though  the  usefulness 
of  inventories  is  not  bound  to  any  one  method  of  constructing  them,  many  of  the  suggested 
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advantages  of  the  current  Air  Force  procedures  stem  from  the  fact  that  incumbents  are  re¬ 
quested  to  provide  information  descriptive  of  the  work  they  do.  The  central  problem  of  the 
presenc  investigation  was  to  explore  some  of  the  characteristics  of  this  information-giving 
behavior  pertaining  to  such  matters  as  the  attributes  of  incumbents  who  provide  the  greatest 
quantity  and  highest  quality  of  information,  the  number  of  incumbents  needed  to  obtain  the 
optimum  amount  of  information,  and  whether  or  not  certain  types  of  work  activity  information 
are  easier  to  obtain  than  others. 

Assume  that  men  from  a  particular  Air  Force  specialty  are  given  a  task  inventory  which 
is  incomplete  and  does  not  include  all  the  tasks  of  their  speciality.  Will  the  men  write  in 
statements  describing  tasks  they  have  done  which  are  not  included  in  the  existing  list?  In 
view  of  individual  differences,  it  seems  certain  that  the  information  yield  from  different  men 
will  vary  in  both  quantity  and  quality.  A  better  way  to  ask  the  question  would  be:  How  large 
a  sample  of  incumbents  is  needed  to  get  a  reasonably  adequate  and  complete  listing  of  all  the 
tasks  performed  in  a  specialty? 

The  question  of  the  incumbent  sample  size  necessary  to  produce  various  yields  of  missing 
task  information  was  central  to  the  present  research.  However,  the  investigation  was  also  de¬ 
signed  to  deal  with  a  number  of  related  problems.  The  issue  of  sample  size  cannot  be  com¬ 
pletely  divorced  from  the  question  of  sample  selection.  Incumbents  will  differ  in  the  extent 
to  which  they  provide  useful  information.  It  is  important  to  know  whether  these  differences 
can  be  reliably  associated  with  other  individual  variables  which  may  subsequently  be  used  to 
select  incumbents  who  will  yield  the  most  and  the  best  job  information. 

Another  purpose  of  the  present  research  was  to  identify  task  variables  related  to  the 
frequency  with  which  tasks  are  written  in  by  incumbents.  Is  a  task  more  likely  to  be  recovered 
if  it  is  difficult  or  easy?  If  it  is  performed  frequently  or  infrequently?  If  doing  the  task  once 
takes  a  long  time  or  a  short  time?  Answers  to  these  questions  are  essential  in  the  design  of 
procedure  statements  which  are  most  efficient  for  use  where  incumbents  provide  work  activity. 

Overview  of  the  Research  Procedure 

Temporally,  the  procedures  of  the  investigation  may  be  divided  into  four  phases.  During 
the  first  phase,  four  typical  specialties  were  selected  for  study:  Ground  Radio  Operator  (29350)/ 
Automotive  Repairman  (47151),  Aircraft  Hydraulic  Repairman  (42152),  and  Accounting  &  Finance 
specialist  (67150).  For  each  specialty  an  initial  inventory  was  prepared  from  standard  Air  Force 
materials  describing  the  specialty.  The  preparation  of  the  inventories  required  numerous  deci¬ 
sions  with  respect  to  the  structure  and  organization  of  task  statements.  Criteria  for  inclusion 
and  exclusion  of  statements  were  developed,  and  an  attempt  was  made  to  organize  statements 
under  general  rubrics  having  applicability  beyond  the  particular  specialties  studied.  Decisions 
during  this  phase  were  facilitated  by  information  from  preliminary  tryouts. 

Three  forms  of  the  inventory  for  each  specialty  were  constructed.  One  of  these,  Form 
100,  contained  all  suitable  statements  gathered  from  standard  sources.  Form  90  was  con¬ 
structed  by  randomly  deleting  10  percent  of  the  statements  in  Form  100.  Similarly,  Form  80 
contained  80  percent  of  the  statements  in  Form  100.  Deletions  were  random,  but  with  the  re¬ 
striction  that  the  same  statement  would  not  be  eliminated  from  more  than  one  form.  The  use 
of  the  three  forms  guaranteed  that  some  of  the  inventories  administered  to  incumbents  would 
be  incomplete.  It  was  further  intended  that  recoveries  of  intentionally  omitted  statements 
could  then  be  used  as  one  basis  for  determining  sample  sizes  necessary  to  produce  inventories 
of  known  completeness. 
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During  the  second  phase  of  the  research,  inventories  were  administered  to  360  incum¬ 
bents  in  each  of  the  four  specialities.  One-third  of  the  men  in  each  specialty  took  Form  100, 
another  third  Form  90,  and  the  remaining  third  Form  80. 

The  sampling  plan  allowed  selection  of  bases  so  as  to  provide  appropriate  geographical 
and  command  representation  among  the  incumbents.  Inventories  were  mailed  to  test  control 
officers  who  arranged  for  and  conducted  all  testing  at  the  bases  selected. 

The  inventories  used  during  the  initial  administration  contained  written  instructions 
directing  incumbents  to  check  all  statements  describing  tasks  they  had  performed  in  the  past 
year.  They  were  further  instructed  to  extend  the  existing  list  by  writing  in  statements  de¬ 
scribing  other  job-related  tasks  they  had  performed  during  that  period. 

During  the  third  phase  of  the  project,  data  collected  from  the  initial  administration  were 
used  to  prepare  a  single  revised  task  inventory  for  each  specialty.  Revisions  were  based 
upon  the  write-ins  of  incumbents.  At  the  same  time,  rating  scales  were  developed  to  be  ad¬ 
ministered  with  the  revised  inventories.  Incumbents  rated  all  tasks  they  had  performed  in 
terms  of  three  dimensions:  frequency  of  performance,  average  time  it  takes  to  do  the  task 
once,  and  the  amount  of  training  and  experience  required  to  perform  the  task. 

Revised  inventories  and  the  rating  scales  were  administered  during  the  fourth  phase  of 
the  research.  This  administration  was  similar  to  the  first  in  that  it  was  accomplished  by 
mail  and  through  test  control  officers.  Approximately  150  men  were  tested  in  each  specialty. 
Some  had  participated  in  the  initial  administration  while  others  had  not. 

2.  PREPARATION  OF  TASK  INVENTORIES 


Selection  of  Specialties 

In  order  to  test  and  establish  the  generality  of  results  obtained  from  the  investigation, 
a  decision  was  made  at  the  outset  to  select  specialties  from  different  career  fields.  The 
choice  of  Ground  Radio  Operator,  Automotive  Repairman,  Aircraft  Hydraulic  Repairman,  and 
Accounting  &  Finance  Specialist  provided  one  operator  job,  one  office  job,  and  two  mainte¬ 
nance  jobs. 

All  data  were  gathered  from  five-level  airmen,  since,  men  above  the  five-level  normally 
become  supervisors.  Five-level  airmen  probably  perform  a  broader  range  of  tasks  than  spe¬ 
cialists  at  other  levels. 

General  Rationale  Used  in  the  Construction  of 
Inventories  for  the  Initial  Administration 

Since  the  research  was  to  be  conducted  using  four  different  specialties,  it  was  neces¬ 
sary  to  decide  whether  to  put  task  inventories  for  different  specialties  into  a  common  form. 
Should  statements  be  similar  in  structure  across  specialties,  and  should  a  single  outline  be 
used  to  prepare  all  inventories?  It  was  decided  to  make  inventories  for  different  specialties 
as  similar  as  possible  so  long  as  no  obvious  injustice  was  done  to  the  "natural”  or  customary 
organization  of  work  within  a  specialty;  it  is  essential  that  an  inventory  appear  sensible  to 
the  men  who  take  it. 

The  decision  to  use  the  same  rules  to  construct  different  inventories  stemmed  from  a 
desire  to  determine  the  generality  of  the  findings  for  particular  specialties.  Though  it  is  pos¬ 
sible  that  quite  different  inventories  could  produce  similar  conclusions  with  respect  to  the 
major  variables  of  the  study  and  thus  establish  maximum  generality,  it  seemed  more  likely 
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that  invariances  in  the  findings  would  occur  if  inventories  were  alike.  Furthermore,  if  inven¬ 
tories  from  different  specialties  were  idiosyncratic  it  might  be  difficult  to  integrate  information 
across  specialties  and  to  serve  the  purposes  of  the  research. 

The  attempts  to  make  some  general  decisions  concerning  the  construction  of  task  inven¬ 
tories  may  be  described  in  two  parts:  (I)  the  definition  of  a  task  statement,  and  (2)  the  organ¬ 
ization  of  task  statements. 

Definition  of  a  Task  Statement 

Dictionary  definitions  of  intelligence  tests  and  intelligence  test  items  would  certainly 
be  of  some  help  to  the  individual  who  wished  to  construct  an  instrument  to  measure  intellectual 
abilities.  However,  he  would  probably  profit  considerably  more  from  a  detailed  description  of 
the  procedures  and  rationale  used  in  the  construction  of  existing  intelligence  tests.  The  sit¬ 
uation  is  similar  when  it  comes  to  constructing  task  inventories.  Most  suggested  definitions 
for  task  statements  are  of  the  dictionary  type.  Structurally,  a  task  statement  is  defined  as 
having  a  subject,  a  verb,  and  an  object.  The  subject  is  the  incumbent  himself  (I  .  .  .  ),  the 
verb  describes  an  action  (have  repaired  .  .  .  ),  and  object  is  the  object  of  the  verb  (wheel  cyl¬ 
inders).  So  a  task  statement  might  read  "1  have  repaired  wheel  cylinders.”  Sometimes  the 
subject  is  left  understood  and  other  tenses  of  the  verb  are  used.  In  content,  a  task  statement 
is  generally  said  to  describe  a  worker  activity  intermediate  in  specificity  between  a  duty  and 
an  element.  A  duty  has  been  defined  as  "a  large  segment  of  the  work  done  by  an  individual” 
(Morsh  el  al.,  1961).  McCormick  &  Tombrink  (I960)  define  a  task  as  "an  activit  composed 
of  small  work  units  or  procedural  steps  that  are  closely  related  to  each  other  and  share  a  com¬ 
mon  goal,  i.e.,  that  are  defined  by  the  task  itself.”  The  same  authors  define  an  element  as 
"one  of  the  procedural  work  steps  involved  in  a  more  general  activity,  e.g.,  a  task.  It  would 
appear  meaningless  for  a  worker  to  perform  an  element  solely  for  the  sake  of  achieving  its 
own  outcome.” 

Definitions  like  these  require  considerable  elaboration  before  they  become  specific 
enough  to  be  of  much  help  in  the  construction  of  a  task  inventory.  In  the  present  investiga¬ 
tion  it  seemed  preferable  to  define  a  task  statement  by  erumerating  some  of  the  large  number 
of  decisions  made  during  the  construction  of  the  inventories.  The  decisions  are  not  regarded 
as  model  ones;  rather,  they  describe  and  rationalize,  to  some  extent,  part  of  the  procedure 
used  in  this  study.  In  many  cases  good  arguments  could  be  offered  to  support  alternative 
procedures.  The  following  is  a  discussion  of  some  of  the  considerations  used  in  arriving  at 
general  decisions  concerning  task  statements  and  the  rationale  for  these  decisions. 

Problem  l 

At  a  very  gross  level  it  is  possible  to  distinguish  the  what  and  the  how  of  the  activities 
men  perform  as  a  part  of  their  jobs.  Should  task  statements  be  concerned  with  both  of  these? 

If  so,  should  they  be  cast  into  single  or  multiple  statements? 

In  many  instances  the  knowledge  of  what  a  man  does  effectively  makes  information  con¬ 
cerning  the  purpose  of  the  method  by  which  the  activity  is  accomplished  redundant.  If  an 
electronic  technician  checks  magnetrons,  there  may  be,  to  all  intents  and  purposes,  only  one 
way  to  perform  the  task  as  well  as  only  one  reason  or  purpose  for  performing  it.  For  instance 
he  may  check  the  tube  to  see  if  it  works.  If  however,  there  is  more  than  one  reason  for  check¬ 
ing,  this  can  be  reflected  by  formulating  two  task  statements  such  as:  checks  gas  pressure 
of  tube;  and  checks  grid  current  of  tube.  The  checking  may  involve  a  single  piece  of  equip¬ 
ment  and  a  single  procedure.  On  the  other  hand,  information  as  to  how  a  task  gets  done  may 
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not  be  implicit  in  the  statement  of  the  task.  The  specialist  may  have  considerable  latitude 
in  the  choice  of  a  method  or  procedure.  Kven  though  a  Ground  Radio  Operator  can  send  mes¬ 
sages  both  by  voice  and  Morse  code,  it  may  be  important  to  know  whether  he  does  in  fact 
use  both  methods  or  how  frequently  he  uses  each.  Thus  a  diversity  of  purposes  as  well  rs 
methods  may  be  expressed  by  formulating  a  number  of  simple  task  statements. 

Granted  that  information  on  method  and  procedures  is  useful,  how  shall  such  information 
be  incorporated  into  task  statement  form?  The  problem  is  made  apparent  by  considering  the 
four  statements  below,  two  concerned  with  what  some  specialist  might  do,  two  concerned  with 
how  the  tasks  are  accomplished. 

WHAT  HOW 

1.  Calculates  correlation  1.  Uses  fully-automatic 

coefficients.  desk  calculator. 

2.  Does  single  classification  2.  Uses  IBM  650  computer 

analyses  of  variance. 

If  one  were  to  use  each  of  these  items  as  the  basis  for  a  separate  and  distinct  task  statement, 
it  is  immediately  obvious  that  some  potentially  useful  information  might  be  lost.  It  may  be 
impossible  to  say  "how  a  man  does  what.”  The  individual  who  "calculates  correlation  co¬ 
efficients”  and  "uses  a  fully-automatic  desk  calculator”  may  or  may  not  "calculate  correla¬ 
tion  coefficients  using  a  fully-automatic  desk  calculator.”  Only  by  including  statements 
similar  to  the  last  one  would  information  on  the  conjunction  of  the  what  and  how  be  obtained. 
Despite  the  inherent  loss  of  information,  it  was  decided  not  to  include  such  statements  in  the 
inventories  prepared  for  this  study.  The  problem  was  a  practical  one  of  the  length  of  the 
inventory.  If  all  sensible  combinations  of  whals  and  hows  were  allowed,  an  inventory  could 
rapidly  become  of  unmangeable  length.  All  possible  combinations  of  just  10  activities  with 
three  methods  would  produce  30  statements.  This  is  contrasted  with  13  statements  if  the  ' 
two  kinds  of  information  arc  kept  separate. 

The  decision  to  make  separate  statements  of  the  what  and  how  of  activities  is  consistent 
with  the  principle  that  statements  be  kept  simple  and  as  free  as  possible  of  qualifying  phrases 
and  clauses.  One  consequence  of  this  decision  for  each  of  the  inventories  was  the  use  of  a 
separate  section  containing  statements  on  the  use  of  tools  and  special  equipment.  So,  for 
example,  one  statement  in  the  Automotive  Repairman  inventory  read,  "I  have  used  micrometers.” 
One  in  the  Accounting  &  Finance  Specialist  inventory  read,  "1  have  operated  printing  calcu¬ 
lators.” 

If  the  Automotive  Repairman  uses  both  micrometers  and  depth  gauges  it  is  probably  of 
minimal  value  to  know  which  measuring  instrument  is  used  in  performing  those  tasks  in  which 
one  or  the  other  is  used,  especially  since  there  are  many  source  materials  such  as  maintenance 
manuals  which  give  instructions  as  to  how  to  perform  each  task.  Although  the  how  may  not  be 
stated  directly  in  the  task  statement,  it  may  often  be  inferred  from  the  list  of  special  tools  and 
equipment  used.  If  there  is  some  special  interest  in  a  topic  of  this  kind,  a  special  purpose 
type  of  task  statement  which  includes  all  the  necessary  qualifying  phrases  and  classes  can 
be  used.  For  most  purposes,  however,  the  simple  type  of  statement  seems  preferable. 

Problem  2 

A  similar  problem  arose  in  connection  with  references  to  the  purposes  or  whys  of  activi¬ 
ties.  It  would  be  possible  to  include  in  task  inventories  statements  such  as:  "I  have  removed 
jet  engines  for  periodic  inspections”;  and  "I  have  calculated  correlation  coefficients  for  factor 
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analyses.”  For  the  most  part  such  statements  were  disallowed.  The  arguments  against  them 
are  similar  to  those  given  in  connection  with  the  bow  of  activities.  Furthermore,  in  many  cases 
an  activity  is  the  same  irrespective  of  the  purpose  for  which  it  is  performed.  To  compute  a 
correlation  coefficient  is  the  same  process,  whether  for  factor  analysis  or  for  some  other  pur* 
pose. 

Frequently  the  what  and  the  why  of  activities  are  difficult  to  discriminate.  The  problem 
is  mixed  in  with  the  question  of  the  specificity  of  tasks.  The  purpose  of  an  act  can  frequently 
be  given  as  the  accomplishment  of  a  more  general  activity.  Performance  of  a  periodic  inspec¬ 
tion  inay  be  the  reason  for  removal  of  a  jet  engine,  but  it  also  is  a  less  specific  (more  general) 
activity.  If  statements  such  as  "1  have  removed  jet  engines  for  periodic  inspections”  are 
regarded  as  undesirable,  should  two  statements  be  formed,  one  reading  "I  have  removed  jet 
engines”  and  the  other  "1  have  performed  periodic  inspections?”  The  two  statements  are  not 
of  the  same  specificity.  They  are  not  independent  in  time,  but  neither  are  they  perfectly  cor¬ 
related.  At  the  present  time  there  is  neither  a  simple  nor  a  completely  satisfactory  answer  to 
the  question  of  whether  both  statements  should  be  included  in  an  inventory.  The  tentative 
answer  which  was  applied  in  the  construction  of  the  inventories  used  in  the  present  investiga¬ 
tion  is  discussed  in  the  next  section  which  deals  more  generally  with  problems  of  the  speci¬ 
ficity  of  statements. 

Problem  3 

As  was  mentioned  earlier,  efforts  to  distinguish  tasks  from  other  worker  activities  have 
focused  on  distinctions  in  terms  of  specificity.  Therefore  the  very  name  "task  inventory” 
implies  that  the  set  of  activities  described  will  be  fairly  homogeneous  on  a  specificity  dimen¬ 
sion.  In  the  initial  stages  of  construction,  considerable  effort  was  devoted  to  developing 
criteria  of  specificity.  Though  some  of  the  criteria  were  helpful  in  the  preparation  of  the 
inventories,  it  soon  became  apparent  that  a  goal  of  homogeneous  specificity  (and  independent 
tasks)  was  not  only  too  constrictive  but  was  also,  in  some  ways,  at  odds  with  the  purpose  of 
the  study. 

Following  are  some  examples  which  illustrate  problems  of  specificity  and  temporal 
independence  of  tasks  which  a  writer  of  inventories  might  face.  The  examples  are  used  as 
vehicles  for  explaining  decisions  and  rules  of  construction  which  characterized  the  present 
investigation. 

Example  l.  The  specificity  of  a  task  statement  can  often  be  judged  in  terms  of  its 
object.  The  statement  "I  have  repaired  fuel  systems”  is  more  general  than  "I  have  repaired 
fuel  pumps.”  Should  both  statements  be  included  in  a  task  inventory?  It  was  decided  not  to 
include  both  statements,  and  an  attempt  was  made  to  maintain  some  equality  of  specificity 
within  the  narrowly  defined  sections  of  the  inventory.  One  section  of  the  inventory  for  Auto¬ 
motive  Repairmen  had  to  do  with  repairing  parts  of  the  fuel  system  (intake  manifolds,  carbu¬ 
retors,  fuel  pumps,  air  cleaners,  governors,  etc.).  A  separate  statement  was  constructed  to 
deal  with  repairing  each  of  the  parts,  but  no  statement  on  "repairing  the  fuel  system”  was 
allowed. 

Example  2.  In  Example  1  it  might  be  granted  that  at  some  low  level  of  organization 
equality  of  specificity  should  be  maintained,  but  it  might  be  asked  why  the  particular  level 
was  chosen.  Wouldn’t  it  have  been  adequate  to  include  statements  about  repairing  the  fuel 
system,  repairing  the  electrical  system,  etc.,  without  going  into  greater  detail?  There  are 
two  answers  to  this  question.  The  first,  probably  of  lesser  importance,  is  that  statements  of 
the  type  included  (I  have  repaired  carburetors,  etc.)  seem  more  consistent  with  the  usual 
definition  of  a  task.  The  second  answer  depends  on  judgments  with  respect  to  the  utility  of 
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the  information.  In  some  applications,  statements  such  as  "1  have  repaired  fuel  systems" 
may  provide  information  exactly  at  the  level  of  specificity  needed.  Under  such  circumstances 
gathering  of  more  detailed  data  by  the  use  of  very  specific  statements  would  be  pointless. 

The  general  conclusion  is  that  there  must  be  knowledge  or  judgments  about  the  specificity  of 
information  which  will  be  useful.  In  the  absence  of  such  knowledge,  an  attempt  should  be 
made  to  pitch  statements  at  the  level  of  specificity  which  will  produce  a  maximum  amount  of 
information  within  the  practical  limitations  set  by  the  ability  and  available  time  of  the  incum¬ 
bents.  Application  of  these  principles  is  illustrated  by  the  use  of  the  statement  "1  have  in¬ 
spected  fuel  systems”  in  the  inventory.  This  task  is  probably  about  equal  in  specificity  to 
"1  have  rt[  aired  fuel  systems”  which  was  rejected  for  being  too  general.  Behind  this  apparent 
contradiction  was  a  judgment  to  the  effect  that  more  specific  information  on  inspection  tasks 
was  of  little  value.  This  decision  was  partly  a  consequence  of  reading  the  printed  materials 
describing  the  job  and  partly  due  to  interviews  with  job  incumbents.  One  main  reason  for 
trying  to  include  statements  of  "equal”  utility  or  importance  rather  than  equal  specificity  is 
that  the  latter  alternative  can  add  appreciably  to  the  total  number  of  statements  in  the  inven¬ 
tory.  If  the  verb  "inspected”  had  been  paired  with  the  same  objects  as  the  verb  "repaired,” 
the  inventory  would  have  been  much  longer.  There  are  practical  limits  to  the  length  of  inven¬ 
tories.  Specialists  cannot  take  unlimited  time  to  complete  them.  Assuming  some  fixed  time 
is  allowed,  stretching  out  the  inventory  in  sections  where  information  is  less  important  dimin¬ 
ishes  the  time  which  can  be  allotted  to  the  sections  in  which  more  detail  is  necessary.  The 
decision,  then,  was  to  strive  for  statements  of  equal  utility  in  providing  information  rather 
than  of  equal  specificity. 


Example  3.  Another  example  illustrates  the  point  that  activities  which  are  central  to  a 
man’s  job  should  be  handled  with  statements  of  greater  specificity  than  activities  which  are 
on  the  fringe  or  periphery.  All  of  the  incumbents  in  the  present  investigation  were  five-level 
airmen.  Men  at  this  level  frequently  have  some  supervisory  responsibilities,  but  traditionally 
supervision  is  a  more  important  part  of  the  job  of  seven-level  airmen.  How  much  emphasis 
should  be  placed  on  supervisory  activities  in  inventories  for  five-level  specialties?  Certainly 
not  as  much  attention  should  be  given  to  these  tasks  as  are  given  them  in  preparing  inventories 
for  specialists  whose  principal  responsibilities  are  supervisory.  The  writer  of  a  task  inventory 
gets  his  initial  idea  as  to  the  relative  importance  of  various  activities  from  standard  Air  Force 
materials  describing  a  specialty.  One  important  feature  of  the  task  inventory  method  is  that 
errors  of  judgment  in  this  regard  are  somewhat  self-correcting.  If  the  tasks  are  rated  with 
respect  to  several  attributes  (time,  frequency,  complexity,  etc.)  by  the  incumbents  to  whom 
the  inventories  are  administered,  errors  of  judgment  with  respect  to  the  specificity  needed  in 
various  sections  of  the  inventory  should  be  easy  to  correct. 


Example  4.  This  example  examines  further  some  questions  of  the  specificity  and  temporal 
independence  of  tasks  raised  in  the  discussion  of  the  u/hat,  how,  and  why  of  activities.  At  a 
very  early  phase  in  the  construction  of  the  four  initial  inventories,  it  was  decided  that  there 
was  no  point  to  striving  for  complete  temporal  independence  of  tasks.  Different  task  statements 
describing  the  very  same  "act”  or  activities  which  overlap  in  time  are  not  inherently  bad. 
Consider  an  "act”  of  a  Ground  Radio  Operator:  he  sends  a  short  message.  When  asked  what 
he  was  doing,  he  says,  "I  just  sent  some  weather  information  to  another  station  about  50  miles 
from  here,  and  at  the  same  time  I  made  a  record  of  the  transmission  in  my  position  log.”  All 
of  the  following  statements  in  the  Ground  Radio  Operator  inventory  might  be  checked  by  this 
man  because  of  the  message  he  sent. 


I  have  handled  point-to-point  traffic. 
I  have  encoded  messages. 

I  have  handled  weather  information. 

I  have  operated  SSB  transmitters. 


I  have  used  automatic  keying  equipment. 
I  have  used  authentication  procedures. 

I  have  transmitted  Morse  code. 

I  have  maintained  radio  position  log. 


7 


If  all  of  these  statements  were  regarded  as  poor  because  of  temporal  overlap  and  some 
differences  in  specificity,  the  implications  are  obvious.  These  problems  could  be  avoided 
only  by  forming  some  complex  conjunction  of  the  bits  of  information  into  a  single  statement 
or  by  eliminating  all  but  one  of  the  statements.  Neither  of  these  alternatives  has  much  to 
recommend  it. 

Problem  4 

Attempts  were  made  in  writing  statements  to  achieve  some  degree  of  grammatical  con¬ 
formity.  All  statements  were  worded  with  a  first  person  subject  and  a  present  perfect  predi¬ 
cate:  "I  have  {verb)  (object)."  Modifying  phrases  were  eliminated  as  much  as  possible.  For 
example,  when  a  statement  in  the  source  materials  was  found  to  read  "checks  fuel  system  for 
cleanliness  and  general  operating  condition"  it  was  changed  to,  "I  have  inspected  fuel  sys¬ 
tems."  Wherever  possible,  statements  in  the  source  materials  describing  knowledge,  under¬ 
standing,  or  general  ability  were  converted  to  the  standard  form.  "Understands  function  and 
operation  of  controls  on  remote  control  units"  was  rephrased  as,  I  have  operated  remote 
control  units.*'  This  change  had  a  purpose  beyond  grammatical  consistency.  Statements  such 
as  "Understands.  ..."  are  practically  impossible  for  an  incumbent  to  rate  on  a  time  or  fre¬ 
quency  scale.  At  the  same  time,  the  best  indication  that  an  incumbent  possesses  the  required 
specific  knowledge  is  that  he  performs  the  task. 

Tasks  me st  have  clear  beginnings  and  clear  endings  if  rating  is  to  be  feasible.  A  general 
effort  was  made  to  cast  statements  into  a  form  in  which  they  would  be  easy  to  rate.  Another 
movement  in  the  dilection  of  grammatical  consistency  is  evidenced  by  the  handling  of  the 
source  statement,  "removes  and  replaces  intake  manifolds  and  exhaust  manifolds.  This 
compound  statement  was  partitioned  into  four  simple  statements: 

I  have  removed  intake  manifolds. 

I  have  removed  exhaust  manifolds. 

I  have  replaced  intake  manifolds. 

I  have  replaced  exhaust  manifolds. 

The  decision  to  split  the  paired  verbs,  remove  and  replace,  was  a  difficult  one.  Reasonable 
arguments  can  be  made  for  leaving  them  together  and  for  separating  them.  In  maintenance 
tasks  removing  and  replacing  could  well  be  regarded  as  a  part  of  the  same  task.  Both  actions 
are  performed  by  the  same  man  as  a  part  of  a  repair  operation.  However,  discussions  with 
specialists  prior  to  the  construction  of  the  initial  inventories  indicated  that  a  linkage  between 
verbs  like  remove,  replace,  and  repair  is  not  at  all  necessary  even  though  it  is  frequently  ap¬ 
propriate.  Since  it  was  not  possible  to  identify  those  tasks  in  which  such  verbs  could  and 
could  not  be  split,  they  were  kept  separate. 

Problem  5 

Statements  about  activities  common  to  more  than  a  single  specialty  were  standardized. 
For  example,  statements  in  source  materials  for  three  of  the  AFSCs  read:  assigns  specific 
tasks  to  individuals”;  "makes  assignments  in  accordance  with  the  abilities  of  individual 
workers”;  and  "assigns  work.”  They  were  all  rephrased  to  read,  "I  have  made  work  assign¬ 
ments.”  This  statement  was  included  in  the  supervisory  section  of  all  four  inventories. 

Organization  of  Task  Statements 

There  is  considerable  latitude  in  the  choice  of  a  method  for  organizing  task  statements. 
At  one  extreme  a  random  arrangement  may  be  used.  At  the  other  extreme  a  very  restrictive 
organization  common  to  all  specialties  might  be  applied.  In  the  present  study  the  alternative 
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of  random  ordering  was  quickly  rejected.  It  was  felt  that  a  systematic  arrangement  of  task 
statements  would  be  far  mote  conducive  to  the  detection  of  missing  information  by  incumbents. 
Kven  withour  a  special  concern  for  write-ins,  organization  should  have  many  benefits.  A  sys¬ 
tematic  arrangement  of  statements  should  reduce  the  time  necessary  for  an  incumbent  to  com¬ 
plete  an  inventor)1.  Furthermore,  to  the  extent  that  there  is  communality  in  the  organization 
from  specialty  to  specialty,  successive  inventories  become  easier  to  write  and  interspecialty 
comparisons  are  easier  to  make. 

The  basic  organizational  format  chosen  was  intended  to  apply  to  all  specialties.  Essen¬ 
tially  the  format  is  an  outline  of  all  functions  performed  by  five-level  airmen.  The  outline 
headings  were  selected  after  a  survey  of  AFM  35-1  and  preliminary  work  with  the  four  special¬ 
ties  used  in  the  present  investigation.  Not  all  of  the  outline  headings  apply  to  all  specialties, 
nor  is  it  assumed  that  they  must  occur  in  a  particular  order.  The  major  divisions  of  the  outline 
and  a  few  explanatory  phiases  and  sentences  follow. 

I.  Inspection.  (Examining;  operational  checks;  tasks  usually  accomplished  in  con¬ 
junction  with  routine  or  preventive  maintainence;  may  inspect  materials, 
equipment,  records.) 

II.  Diagnosis.  (Interpretation;  evaluation;  troubleshooting;  analytical  tasks.  Tasks 
with  an  emphasis  on  cognition  and  problem  solving.  Examples:  map  inter¬ 
pretation,  troubleshooting,  interpretation  of  interview  material,  interpretation 
of  data.) 

III.  Service.  (As  in  "service”  station.  Routine  maintainence;  preventive  maintainence; 

the  "keeping  up  of  .  .  . 

A.  Records. 

B.  Equipment. 

C.  Personal  service.  (Steward;  clerking;  providing  information  or  direction 

to  travelers.) 

IV.  Correction.  (Corrective  maintenance;  repair;  elimination  of  malfunctions  and 

errors.) 

V.  Construction.  (Building;  assembly;  fabrication;  installation;  finishing;  collection 
and  preparation  of  materials;  routine  building  as  opposed  to  creative  work.) 

VI.  Design.  (Tasks  requiring  creative  work;  designing  or  modification  of  equipment, 
materials,  and  procedures;  original  writing.) 

VII.  Supervision. 

A.  Work  Arrangements.  (Planning;  scheduling;  organizing  or  routing  work; 

making  work  assignments.) 

B.  Training  and  Guidance.  (Classroom  and  OJT  instruction;  personal 

counseling.) 

C.  Records,  Reports,  and  Correspondence.  (Of  a  supervisory  or  administrative 

nature.) 

VI 11 .  Learning.  (Studying;  OJT;  technical  school  attendance.) 

IX.  Vigilance.  (Observing;  monitoring;  watch-keeping.) 

X.  Use  of  Tools  and  Special  Equipment. 

XI.  Processing.  (Processing  of  information,  records,  food;  to  put  through  a  stage  of 
development;  to  transform;  to  complete;  to  handle.  Examples:  transmitting 
weather  information,  cutting  meat,  typing  a  manuscript.) 

XII.  Special  Skills  and  Procedures.  (Examples:  use  of  Morse  code;  use  of  authentica¬ 
tion  procedures.) 

The  major  divisions  of  each  of  the  four  original  inventories  are  fairly  easy  to  associate 
with  die  headings  of  the  above  outline.  The  inventories  were  divided  into  sections.  For 
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example.  Section  I  in  the  original  Automotive  Repairman  inventory  »,is  introduced  as  follows: 
"This  section  contains  statements  about  routine  or  scheduled  inspections  and  operational 
checks  of  vehicles  anti  auxiliary  equipment.”  The  beginning  of  Section  II  reads:  "This 
section  contains  statements  about  routine  maintenance  and  servicing  of  vehicles,  auxiliary 
equipment,  and  records.”  These  two  introductory  statements  correspond,  respectively,  to  the 
first  anti  third  headings  in  the  outline. 

In  addition  to  the  grouping  of  statements  into  major  sections,  further  divisions  were 
sometimes  made  among  statements  within  sections.  Thus  for  the  Automotive  Repairman  inven¬ 
tory,  statements  in  Section  IV  (corrective  maintenance)  were  subcategorized  into  Part  A, 

Engine;  Part  B,  Fuel  System;  Part  C,  Electrical  System;  etc. 

The  bases  upon  which  major  sections  were  subdivided  were  only  partly  consistent  from 
specialty  to  specialty,  and  were  necessarily  somewhat  arbitrary.  Categorization  depended 
upon  a  knowledge  of  the  specialty  gained  from  the  source  materials.  An  attempt  was  made  to 
group  tasks  in  a  way  which  would  be  regarded  as  "natural”  by  incumbents.  In  the  Automotive 
Repairman  inventory,  for  instance,  tasks  in  the  corrective  maintenance  section  could  have 
been  grouped  under  verb  headings  according  to  the  sort  of  action  performed,  such  as:  Part  A, 
Removal;  Part  B,  Cleaning;  Part  C,  Adjusting;  etc.  Eor  this  AESC,  it  was  felt  that  object¬ 
grouping  would  be  most  appropriate  because  the  repairmen  themselves  conceptualize  their 
jobs  in  terms  of  the  equipment  worked  on,  rather  than  the  kind  of  action  performed  on  the 
equipment.  Verbs  describing  actions  became  the  basis  for  a  third  level  of  organization  in  the 
Automotive  Repairman  inventory. 

It  was  stated  earlier  that  well-organized  as  opposed  to  randomly  ordered  statements 
should  stimulate  write-ins.  Two  further  comments  may  be  made  on  this  point.  First,  an  attempt 
was  made  to  stimulate  write-ins  not  only  by  a  logical  organization  of  the  material  but  by  an 
"on-the-page”  organization.  Spaces  were  provided  for  write-ins  at  the  end  of  each  group  of 
tasks.  This  meant  that  there  were  spaces  on  almost  every  page  of  all  four  inventories.  When 
an  incumbent  contributed  a  statement,  he  did  not  need  to  write  "I  have”;  at  most,  he  had  to 
write  a  verb  and  a  noun,  sometimes  only  a  noun  because  the  verb  was  provided.  Thus,  in  the 
section  on  corrective  maintenance  having  to  do  with  adjusting  parts  of  the  electrical  system, 

spaces  for  write-ins  below  this  section  begin  with  the  words  "I  have  adjusted  _ .” 

Another  implication  of  the  organization  of  task  statements  is  particularly  relevant  to  types  of 
write-ins  received.  If  an  Aircraft  Hydraulic  Repairman  were  working  in  the  section  of  his 
inventory  having  to  do  with  corrective  maintenance  of  power  systems,  he  would  find  a  series 
of  statements  having  to  do  with  the  disassembly  of  reservoirs,  accumulators,  pumps,  pressure 
transmitters,  etc.  The  very  next  section  of  statements  has  to  do  with  the  repair  of  the  very 
same  components.  Still  other  statements  refer  to  additional  actions  on  these  components.  It 
seems  highly  likely  that  if  a  statement  like  "I  have  repaired  accumulators,”  were  left  out  of 
the  inventory,  its  probability  of  recovery  would  be  higher  than  other  statements  for  which  there 
were  not  quite  so  many  cues  to  jog  the  incumbent’s  memory. 

Construction  of  Task  Inventories 

Preliminary  forms  of  the  task  inventories  were  constructed  for  each  of  the  four  specialties 
selected.  These  forms  were  tried  out  and  revised  in  the  preparation  of  the  80-percent,  90- 
percent,  and  100-percent  complete  forms  for  the  initial  administration. 
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Automotive  Repairman,  47/5/ 

The  task  statements  for  this  specialty  were  extracted  from  the  following  sources: 

AF  Manual  35-1,  1  March  1956,  47-11  to  47-13 
On  the  Job  Training,  JP  47151,  June  1956 
Job  Training  Standard,  June  1956. 

In  the  first  / preliminary  form  the  task  statements  were  organized  under  major  headings 
such  as  I.  Determination  of  general  condition  of  vehicle  and  associated  equipment;  II.  Mainte¬ 
nance  of  vehicle  and  auxiliary  equipment.  This  second  heading  was  further  subdivided  into  »■ 
components  such  as  A.  Engine.  H.  Fuel  System,  C.  Electrical  System.  Each  subdivision 
was  arranged  in  the  form  of  a  matrix  with  verbs  such  as  inspected  imd  tested,  cleaned,  removed, 
placed  across  the  top,  and  parts  such  as  cylinder  beads,  crankshaft,  camshaft,  placed  along 
the  side. 

Each  cell  of  the  matrix  contained  a  small  box  in  the  upper  right-hand  corner  which  the 
incumbent  was  directed  to  check  if  he  had  performed  the  task  during  the  previous  three  months. 
Also  within  each  cell,  spaces  were  available  for  making  three  ratings  for  those  tasks  which 
had  been  performed. 

A.  The  frequency,  in  terms  of  number  of  times  per  day,  week,  or  month. 

B.  The  average  time  in  terms  of  hours,  minutes,  or  days. 

C.  The  training  preparation  which  the  respondent  judged  necessary  to  perform  the  task 

proficiently,  in  terms  of  the  percentage  of  present  level  of  skill  attained  at  the 
completion  of  technical  school  training. 

After  each  section,  space  was  provided  for  writing  in  tasks  which  the  respondent  had 
performed  but  which  were  not  listed  from  the  original  sources.  These  write-in  statements  were 
to  be  rated  in  the  same  way  as  the  provided  statements. 

This  preliminary  form  was  tried  out  on  five-level  Automotive  Repairman  job  incumbents. 
While  in  general  the  respondents  were  able  to  check  and  rate  the  items'in  a  satisfactory  manner, 
very  little  in  the  way  of  write-in  statement  yield  was  obtained.  This  was  probably  due  to  the 
large  amount  of  time  and  attention  required  to  go  through  the  listed  statements  and  to  check 
and  rate  them.  Also  the  "difficulty”  rating  dimension  was  not  entirely  satisfactory  because 
it  was  not  clear  whether  the  specialists  were  making  their  ratings  with  reference  to  time,  phys¬ 
ical  effort,  or  amount  of  skill  required.  The  matrix  form  of  presentation  of  the  task  statements 
was  not  completely  satisfactory  since  the  entire  matrix  could  be  considered  at  once  rather  than 
each  statement  individually;  sometimes  the  respondent  would  skip  over  an  entire  section  when 
he  might  have  checked  some  of  the  items  in  the  section  if  they  had  been  presented  independently. 

The  inventory  was  revised  and  tried  out  again  on  job  incumbents.  In  this  revision  the 
statements  were  organized  in  the  same  overall  outline  as  in  the  first  form  but  they  were  arranged 
within  subsections  serially,  rather  than  in  matrix  form,  so  that  the  task  statements  would  be 
considered  one  at  a  time.  In  checking  each  task  statement  the  respondent  indicated  whether 
he  had  performed  the  task  during  the  previous  3  to  6  months  or  6  to  12  months.  This  scale 
was  used  to  obtain  some  indication  of  the  time  period  it  would  be  best  to  use  for  the  first  ad¬ 
ministration  form.  The  frequency  and  average-time  rating  dimensions  were  similar  to  those 
used  in  the  previous  form.  The  difficulty  dimension  was  changed  to  a  4-point  scale:  0,  not  at 
all;  1,  slightly;  2,  moderately;  3,  very  difficult.  The  incumbent  was  to  indicate  the  nature  of 
the  difficulty  by  checking  under  one  of  the  following:  Training  or  Experience,  Complex, 
Monotonous,  Heavy  Work ,  Rushed ',  Working  Conditions.  Space  was  provided  for  any  necessary 
comments  to  explain  the  responses  or  to  indicate  questions  concerning  the  statements.  Again 
in  the  administration  of  this  form,  although  it  was  satisfactory  in  most  other  respects,  few 
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write-ins  were  obtained.  Since  a  major  purpose  of  the  research  was  to  assess  the  feasibility 
of  obtaining  information  directly  from  the  incumbents  by  means  of  write-ins,  it  was  felt  that  a 
revision  of  the  format  that  would  stimulate  a  greater  number  of  write-ins  was  needed.  One 
reason  for  the  small  yield  of  write-ins  was  probably  the  amount  of  attention  and  effort  that  was 
directed  toward  checking  and  rating  the  listed  statements.  A  revision  of  the  format  that  put 
more  emphasis  on  writing  in  and  less  on  rating  and  checking  was  needed  to  increase  the  number 
of  write-ins  obtained. 

The  format  of  the  Automotive  Repairman  Task  Inventory  that  was  adopted  for  the  initial 
administration  was  similar  to  the  second  preliminary  form,  except  that  the  three  rating  dimen¬ 
sions  were  dropped  from  this  form.  It  was  not  necessary  to  obtain  the  ratings  of  the  items 
during  the  initial  adminisuation  since  the  results  of  the  ratings  were  not  needed  until  the 
analyses  in  connection  with  the  second  administration.  A  change  in  the  directions  for  the 
initial  administration  form  was  made  to  stimulate  more  writing  in.  The  incumbent  was  directed 
to  go  through  the  following  three  steps  in  the  order  listed: 

(1)  Read  each  task  in  the  inventory. 

(2)  Write  in  all  missing  tasks  you  have  done  during  the  past  year. 

(3)  Check  statements  "yes”  or  "no.” 

Having  the  respondent  look  through  the  booklet  and  write  in  statements  which  were  not  listed 
before  he  indicated  whether  or  not  he  had  performed  the  listed  task  during  the  preceding  year, 
directed  a  great  deal  more  attention  to  the  writing-in  process.  The  importance  of  the  writing-in 
step  was  emphasized  in  the  directions  both  to  the  respondent  and  to  the  testing  officer.  The 
results  of  this  change  of  format  and  emphasis  was  to  produce  a  larger  number  of  write-ins  for 
the  specialty  as  a  whole,  although  there  was  a  large  range  of  individual  differences  in  the 
number  of  write-ins. 

Accounting  0  Finance  Specialist,  67J50 

Three  preliminary  forms  were  constructed  for  this  specialty.  In  the  first  preliminary 
form  the  task  statements  were  obtained  from  the  following  sources: 

AF  Manual  35*1 ,  1  March  1956,  67-9  to  67-10a 

On  the  Job  Training,  JP  67150,  September  1958 

Job  Training  Standard,  September  1958 

Specialty  Description,  September  1958. 

The  task  statements  were  grouped  by  verb  such  as  /  have  prepared,  I  have  maintained,  and 
under  each  verb  the  forms  and  procedures,  such  as  vouchers,  reports,  schedules,  were  grouped. 
The  respondent  was  directed  to  check  the  last  time  he  had  performed  the  task: 

A.  within  the  past  3  months, 

B.  3  months  to  a  year  ago, 

C.  more  than  a  year  ago, 

D.  never  while  at  this  skill  level. 

He  was  also  asked  to  rate  the  number  of  minutes  needed  to  complete  the  task,  on  the  average, 
and  either  the  number  of  times  the  task  was  done  per  average  week,  or  per  3-month  period.  He 
was  directed  at  the  end  of  each  list  to  look  over  the  listed  statements  and  to  write  in  and  rate 
tasks  he  had  done  but  which  were  not  listed  in  that  section.  Space  was  provided  for  comments 
where  necessary  either  to  clarify  the  statements  or  his  responses.  This  form  was  tried  out  on 
a  sample  of  incumbents  and  it  was  found  that  the  format  and  directions  were  clear,  but  that 
only  a  small  yield  of  write-in  statements  was  obtained.  From  interviews  with  the  specialists, 
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it  was  learned  that  even  though  accounting  and  finance  are  listed  under  one  specialty  they 
had  in  fact  not  been  merged  and  individual  specialists  tended  to  work  in  one  area  or  the  other. 

The  second  preliminary  form  benefited  from  the  administration  of  the  first  form  and  also 
from  experience  with  the  inventory  for  the  other  specialties.  The  rating  dimensions  were 
dropped  and  more  attention  was  directed  toward  obtaining  write-ins.  The  respondent  first 
looked  through  the  inventory  without  making  any  response  on  it;  secondly  he  wrote  in  the  tasks 
that  he  had  done  within  the  past  year  but  which  were  not  indicated  in  the  appropriate  section; 
and  thirdly  he  checked  the  task  statements  he  had  performed  in  the  past  year. 

The  first  administration  form  of  the  Accounting  &  Finance  Task  Inventory  was  like  the 
second  preliminary  tryout  form  except  that  descriptions  were  inserted  in  the  beginning  of  each 
section  to  make  clear  the  kind  of  tasks  included  and  to  be  written  in  that  section. 

Ground  Radio  Operator,  29350 

Two  preliminary  forms  of  the  Ground  Radio  Operator  Task  Inventory  were  tried  out.  The 
task  statements  were  extracted  from  the  following  sources: 

AF  Manual  35-1,  1  March  1956,  29-21 

On  the  Job  Training,  JP  29350,  November  1958 

Job  Training  Standard,  November  1958 

Airman  Proficiency  Test  Outline,  1  September  1957. 

In  the  first  form  statements  were  classified  by  headings  such  as  Types  of  Messages  ( Content ), 
Types  0/  Messages  (Sources  and  Destination ),  and  Ground  Radio  Procedures.  Within  each 
section  the  statements  were  grouped  by  verbs  such  as  /  have  transmitted,  I  have  relayed,  1 
have  received.  The  respondent  was  to  indicate  whether  or  not  he  had  performed  the  task  with¬ 
in  the  past  year  and  to  comment  either  on  his  responses  or  on  the  statements  where  necessary. 
In  addition  to  checking  the  statements,  the  respondent  was  directed  to  yrrite  in  under  each 
section  other  tasks  he  had  done  which  were  appropriate  to  that  section  but  which  were  not 
listed. 

As  a  result  of  the  tryout  of  the  first  preliminary  form,  a  second  preliminary  form  was 
constructed  which  was  similar  to  the  first  one,  except  that  more  categories  were  added  and 
statements  were  introduced  at  the  beginning  of  each  section  to  clarify  the  nature  and  scope 
of  the  tasks  in  that  section.  The  first  administration  form  of  the  Ground  Radio  Operator  Task 
Inventory  was  similar  to  the  second  preliminary  form  except  for  minor  revisions. 

Aircraft  Hydraulic  Repairman,  42152 

Two  preliminary  forms  of  task  inventory  were  constructed  and  tried  out  for  this  specialty. 
The  statements  in  the  first  form  wgre  obtained  from  the  following  sources: 

AF  Manual  35-1,  1  March  1956,  42-19,  42-20 

On  the  Job  Training,  JP  42152,  January  1956 

Job  Training  Standard,  April  1957. 

The  task  statements  were  organized  under  major  headings  such  as  Performs  preventive 
maintenance  on  Aircraft  Hydraulic  Systems,  Performs  corrective  maintenance  on  Aircraft 
Hydraulic  Systems.  Each  of  these  major  headings  was  broken  down  into  subsystems  such  as 
A.  Power  System,  B.  Flight  Control  System,  C.  Actuating  System.  The  task  statements  were 
then  grouped  by  verbs  such  as  I  have  inspected,  I  have  cleaned,  I  have  adjusted,  and  the  re¬ 
spondent  indicated  whether  or  not  he  had  performed  the  task  within  the  past  year.  At  the  end 
of  each  set  of  statements  space  was  provided  to  write  in  other  statements  that  belonged  in 
the  section  but  were  not  listed. 
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The  second  preliminary  form  was  like  the  first  except  for  revisions  made  as  a  result  of 
the  initial  tryout  and  for  placing  a  statement  at  the  beginning  of  each  section  to  clarify  the 
nature  and  scope  of  the  tasks  included  within  that  section.  The  first  administration  form  of 
Aircraft  Hydraulic  Repairman  Task  Inventory  was  similar  to  the  second  preliminary  form. 

}.  INITIAL  ADMINISTRATION 


Sampling 

The  plan  for  the  initial  administration  called  for  170  copies  of  each  of  the  three  forms 
to  be  sent  to  incumbents  in  each  AFSC.  A  distribution  of  frequency  of  incumbents  by  com¬ 
mand,  base,  and  AFSC  was  obtained  from  the  major  air  commands  to  be  sampled.  The  sampling 
was  set  up  to  be  proportional  by  commands.  Consideration  was  also  given  to  having  the  north¬ 
ern  and  southern  climatic  regions  represented  approximately  equally.  It  was  considered  desir¬ 
able  to  have  all  the  specialists  of  a  given  AFSC  stationed  at  a  given  base  included  in  the 
sample  to  avoid  any  bias  that  might  be  introduced  by  selection  of  personnel  at  the  base  level. 

Sampling  was  random  within  the  restrictions  imposed  by  commands,  AFSCs,  and  climatic 
conditions.  For  the  Ground  Radio  Operator  specialty,  it  was  impossible  for  half  of  the  incum¬ 
bents  to  be  drawn  from  each  of  the  two  climatic  regions  in  Air  Training  Command,  since  almost 
all  of  its  radio  operators  were  at  one  base  in  the  southern  climatic  region,  and  the  number  at 
that  base  alone  exceeded  the  total  number  needed  from  that  command. 

Summaries  of  the  distribution  of  incumbents  by  command  and  climatic  region  for  each 
specialty  are  shown  in  Table  1.  The  numbers  in  the  column  at  the  right  indicate  the  frequencies 


Table  l.  Distribution  of  Incumbent  Samples  by  Command 
and  Climatic  Region 


Specialty 

Command 

Climatic 

South 

Region 

North 

Obtained 

Total 

Propor¬ 
tional  N 

29350 

SAC 

71 

212* 

283 

312 

Ground 

TAC 

156 

38* 

194 

165 

Radio 

ATC 

37 

0 

37 

33 

Operator 

Total 

264 

250 

514 

510 

67150 

SAC 

129 

139 

268 

282 

Accounting 

TAC 

48 

48* 

96 

90 

3 l  Finance 

ATC 

82 

64* 

146 

138 

Specialist 

Total 

259 

251 

510 

510 

42152 

SAC 

208 

173 

381 

384 

Aircraft 

TAC 

27 

39 

66 

63 

Hydraulic 

ATC 

32 

33 

65 

63 

Repairman 

Total 

267 

245 

512 

510 

47151 

SAC 

123 

161 

284 

294 

Automotive 

TAC 

65 

66* 

131 

117 

Repairman 

ATC 

57 

43 

100 

99 

Total 

245 

270 

515 

510 

*AU  northern  bases  of  command  included. 
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needed  in  the  .sample  to  make  its  frequencies  proportional  to  the  distribution  of  the  specialty 
in  the  commands.  Since  the  sampling  was  done  by  random  selection  of  bases,  rather  than 
individuals,  the  obtained  frequencies  are  not  exactly  equal  to  the  proportional  frequencies, 
but  as  close  as  could  be  obtained  within  the  sampling  restrictions.  The  departures  from  pro¬ 
portional  representation  by  commands  are  in  the  direction  of  the  more  equal  representation  of 
commands  rather  than  in  the  opposite  direction. 

The  sampling  was  done  separately  for  each  specialty.  In  each  case  the  sampling  was 
done  first  in  the  Air  Training  Command  and  the  Tactical  Air  Command  since  these  commands 
had  the  most  limited  distributions  by  climatic  regions,  and  then  the  sampling  was  completed 
in  the  Strategic  Air  Command  with  an  effort  to  balance  the  distribution  by  climatic  regions. 

Administrative  Procedures 

Instructions  and  materials  were  mailed  to  the  test  control  officers  who  administered  the 
task  inventories  at  the  bases.  In  order  to  perfect  the  procedure,  an  initial  draft  of  all  corre¬ 
spondence  and  other  materials  was  sent  to  the  test  control  officer  at  a  large  base  in  the  Air 
Training  Command  for  his  evaluation  regarding  clarity,  appropriateness,  etc.  These  materials 
included: 

A  cover  letter  explaining  the  nature  of  the  project, 

Project  Officer’s  Report  on  Administration  of  Task  Inventory, 

Information  Sheet  on  Specialists, 

Sample  copies  of  the  task  inventories. 

A  preliminary  letter  was  sent  to  the  test  control  officer  at  each  base  included  in  the 
sample  to  explain  the  purpose  of  the  research  and  to  determine  (a)  his  correct  mailing  address, 
(b)  whether  the  required  number  of  specialists  would  be  available  during  the  testing  period, 
and  (c)  whether  the  time  period  allowed  for  the  testing  was  satisfactory.  A  copy  of  a  letter 
from  Command  Headquarters  authorizing  administration  of  the  task  inventory  was  inclosed. 
Replies  to  the  letters  indicated  that  in  most  cases  fewer  specialists  were  available  for  testing 
than  were  listed  in  the  initial  distribution.  However,  the  number  available  was  satisfactory 
in  all  specialties  except  that  the  number  of  Ground  Radio  Operators  in  Tactical  Air  Command 
was  less  than  needed.  This  deficiency  was  remedied  by  increasing  the  sample  of  Ground 
Radio  Operators  in  Strategic  Air  Command.  The  number  of  inventories  sent  out  and  the  number 
of  completed  inventories  returned  were  as  follows: 


AFSC 

Sent  Out 

Returned 

Ground  Radio  Operator 

549 

366 

Accounting  &  Finance  Specialist 

510 

394 

Aircraft  Hydraulic  Repairman 

580 

393 

Automotive  Repairman 

553 

398 

Nonreturns  were  due  to  leave,  sickness,  special  detail,  transfer,  or  temporary  duty.  Each  test 
control  officer  returned  the  inventories,  submitted  a  report  on  time  and  place  of  administration, 
indicated  any  special  conditions  associated  with  the  administration,  and  prepared  an  informa¬ 
tion  sheet  on  each  specialist.  The  information  sheet  contained  an  incumbent’s  Airman  Profi¬ 
ciency  Test  scores  and  aptitude  indexes.  In  those  cases  where  a -man  was  not  available  for 
testing,  it  gave  the  reason. 

Scoring  Returns  from  the  Initial  Administration 

The  returns  were  scored  for  two  purposes.  First,  it  was  necessary  to  obtain  measures 
of  the  yield  of  information  from  individuals  and  groups.  Secondly,  a  revised  form  of  the  inventory 
for  each  of  the  four  specialties  was  needed  for  the  second  administration. 
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The  scoring  form  for  a  particular  specialty  consisted,  essentially,  of  all  statements  on 
the  original  100-percent  form  of  the  inventory  plus  a  collection  of  new  statements  derived  from 
the  write-ins  of  incumbents.  No  reorganization  of  the  original  inventories  was  attempted  on 
the  scoring  forms.  There  was  never  an  instance  in  which  an  old  statement  was  revised  on  the 
basis  of  new  information  obtained  from  incumbents.  To  have  done  so  would  have  destroyed 
the  sharp  demarcation  between  old  statements,  omitted  statements,  and  new  statements,  and 
the  problems  of  scoring  the  write-ins  of  individual  incumbents  would  have  been  increased  im¬ 
measurably.  On  the  other  hand,  it  was  possible  to  let  the  revised  inventories  benefit  from  the 
organizational  changes  which  seemed  reasonable  in  view  of  write-ins  obtained  from  incumbents. 
One  of  the  most  important,  but  not  so  obvious,  benefits  from  the  write-in  method  is  that  returns 
suggest  reorganization  as  well  as  additions  to  an  earlier  form  of  an  inventory. 

The  raw  material  available  for  the  preparation  of  the  scoring  form  for  a  particular  inven¬ 
tory  consisted  of  all  the  information  written  in  the  inventory  booklets  by  the  men  sampled  in 
a  specialty.  A  good  deal  of  this  material  is  useless  information.  To  illustrate,  the  360  Ground 
Radio  Operators  provided  a  total  of  186  different  new  statements.  The  186  statements  were 
written  in  a  total  of  1259  times,  an  average  of  slightly  under  7  times  per  statement.  In  addition 
to  the  useful  information  obtained  from  incumbents,  there  were  several  thousand  write-ins  which 
were  not  useful  for  a  variety  of  reasons.  One  of  the  principal  problems  in  scoring  an  inventory 
is  separating  out  the  wheat  from  the  considerable  amount  of  chaff.  The  useless  information 
takes  a  variety  of  forms.  Incumbents  sometimes  write  in  statements  which  are  already  contained 
in  another  part  of  an  inventory.  They  frequently  provide  information  which  is  too  general  or  too 
specific.  Occasionally  write-ins  are  uninterpretable  phrases  or  words. 

The  following  general  procedure  was  used  in  the  preparation  of  the  scoring  forms  for  each 
specialty.  Initially  the  individual  booklets  were  inspected  and  all  phrases,  possible  statements, 
and  comments  were  numbered.  As  each  booklet  was  considered,  a  list  of  potential  new  state¬ 
ments  was  simultaneously  prepared.  This  list  was  organized  like  the  original  inventory.  Very 
little  critical  screening  was  done  in  the  preparation  of  this  list,  but  a  statement  from  a  booklet 
was  not  added  to  the  list  if  it  was  obviously  nonsense,  if  it  was  a  quote  of  a  statement  appear¬ 
ing  on  the  original  inventory,  or  if  it  was  a  duplication  of  a  statement  previously  added  to  the 
list.  The  extraction  of  the  statements  was  done  by  a  single  individual.  The  result  of  the 
extraction  was  a  pool  of  write-in  information  organized  approximately  in  the  form  of  the  original 
inventory  but  with  overlapping  meanings. 

Some  of  the  major  problems  encountered  were: 

(1)  Some  statements  were  at  levels  of  specificity  judged  to  be  inappropriate  for  the  inven¬ 
tory.  Some  were  too  general;  some  far  too  detailed. 

(2)  Incumbents  used  terms  and  phrases  which  required  interpretation  by  subject-matter 
experts.  In  a  number  of  instances,  it  was  not  possible  to  clarify  ambiguities  even  with  expert 
help. 

(3)  Many  contributions  of  incumbents  were  in  a  grammatical  form  different  from  that  used 
for  statements  in  the  inventory. 

(4)  Write-ins  overlapped  one  another  in  meaning  and  overlapped  with  items  on  the  100-  • 
percent  forms. 

(5)  Sometimes  the  statements  of  incumbents  did  not  fit  readily  into  the  organizational 
scheme  of  the  original  inventory. 

To  clarify  the  meanings  of  statements  containing  ambiguous  and  unfamiliar  terms,  members 
of  the  project  interviewed  officers  and  enlisted  personnel  familiar  with  the  four  specialties.  It 
then  became  the  responsibility  of  one  project  member  who  was  highly  familiar  with  the  specialty 
in  question  to  prepare  a  first  draft  of  the  scoring  form  from  the  original  100-percent  form  and 
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the  list  of  potential  new  statements.  Individuals  preparing  these  scoring  forms  treated  the 
lists  of  extracted  statements  much  as  the  source  materials  had  been  treated  in  the  preparation 
of  the  original  inventories.  The  raw  materials  for  new  statements  were  frequently  combined 
and  reworded.  However,  new  material  was  never  combined  with  statements  which  had  origi¬ 
nally  appeared  on  the  100-percent  inventory  of  the  initial  administration.  After  a  draft  of  the 
scoring  form  had  been  prepared  by  one  project  member,  a  second  project  member  made  an 
independent,  critical  review  of  the  statements.  As  many  points  of  disagreement  as  possible 
were  resolved  in  conference.  Differences  still  outstanding  were  resolved  in  conference  with 
a  third  staff  member  after  he  had  made  independent  judgments  concerning  the  points  involved. 

The  scoring  of  the  booklets  of  the  individual  incumbents  became  relatively  routine  in 
view  of  the  procedure  used  in  the  preparation  of  the  scoring  form.  A  statement  on  any  particu¬ 
lar  inventory  was  tagged  if  it  was  regarded  as  sufficient  to  have  prompted  inclusion  of  any  of 
the  statements  in  the  scoring  form.  In  other  words,  it  was  not  necessary  for  a  statement  appear¬ 
ing  in  an  incumbent's  booklet  to  be  identical  to  a  statement  on  the  scoring  form  for  the  incum¬ 
bent  to  receive  credit  for  a  write-in. 

The  result  of  the  scoring  was  an  individual-by-statement  matrix  for  each  specialty.  The 
statements  consisted  of  all  new  statements  on  the  scoring  form  plus  statements  omitted  for 
any  of  the  subgroups.  Entries  in  the  matrices  were  0  or  1,  a  one  indicating  that  a  particular 
incumbent  yielded  a  particular  statement.  These  matrices  provided  the  raw  data  for  the  calcu¬ 
lation  of  values  of  the  criterion  variables  for  several  of  the  analyses. 

Group  Analyses 

In  order  to  determine  the  relationships  between  the  yield  of  information  obtained  by  the 
incomplete  task  inventory  method  and  several  variables  describing  groups  and  individuals,  a 
series  of  multiple  regression  analyses  was  performed.  The  first  set  of  analyses  examined  the 
relationship  between  three  measures  of  the  yield  of  job  information  and  various  characteristics 
of  a  g roup,  such  as  its  size,  specialty,  and  the  form  and  completeness  of  the  inventory  used 
by  the  members  of  the  group.  These  analyses  are  referred  to  as  the  group  analyses. 

A  second  set  of  the  analyses  sought  to  determine  the  relationships  between  the  amount 
of  job  information  yielded  by  an  individual  and  various  characteristics  of  the  individual,  such 
as  his  specialty,  education,  age,  rank,  aptitude  indexes,  Airman  Proficiency  Test  score,  and 
the  length  of  time  he  had  been  at  the  five-level  of  skill.  This  set  of  analyses  is  referred  to 
as  the  individual  analyses. 

The  36  groups  of  incumbents  used  in  the  group  analyses  were  defined  by  all  possible 
combinations  of  the  four  AFSCs  (Ground  Radio  Operator,  Automotive  Repairman,  Aircraft 
Hydraulic  Repairman,  Accounting  St  Finance  Specialist),  three  check  list  forms  (Form  80, 

Form  90,  Form  100),  and  three  sample  sizes  (Size  20,  Size  40,  Size  60).  Data  were  obtained 
from  a  total  of  1440  men,  360  cases  selected  from  the  returns  in  each  of  the  four  specialties. 

Of  the  360  men  within  a  given  specialty,  120  completed  one  of  the  three  forms.  To  create  the 
36  groups  used  in  the  present  analysis,  samples  of  size  20,  40,  and  60  were  drawn  from  each 
of  the  groups  of  120  men  who  took  a  particular  form  of  a  task  inventory.  Each  individual  sam¬ 
ple  was  a  random  nonreplacement  sample.  For  example,  a  sample  of  size  20  from  the  Ground 
Radio  Operators  who  took  Form  80  contained  20  different  men.  It  was  possible,  however,  for 
the  same  men  to  be  included  in  different  samples.  Thus,  a  Ground  Radio  Operator  who  took 
Form  80  and  was  selected  for  a  sample  of  size  20  could  also  be  selected  as  a  member  of  a 
sample  of  size  40. 
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Once  a  single  sample  had  been  selected  for  each  of  the  36  groups,  two  additional  repli¬ 
cations  were  made.  The  sampling  operations  used  for  the  second  and  third  replications  were 
identical  with  those  used  with  the  first.  In  each  new  replication  there  were  no  restrictions 
rhat  depended  on  the  outcome  of  previous  sampling.  Though  all  of  the  group  analyses  could 
have  been  accomplished  with  a  single  replication,  preliminary  work  indicated  the  value  of  the 
additional  samples.  With  a  single  replication  there  were  occasional  reversals  in  the  functions 
relaring  yield  to  sample  size.  At  times,  due  to  sampling  fluctuation,  a  large  sample  produced 
a  smaller  yield  than  a  small  sample. 

Three  dependent  variables  were  used  in  the  group  analyses.  All  three  measures  were 
derived  from  the  acceptable  write-ins  of  individual  incumbents.  There  were  rwo  classes  of 
write-ins:  recovered  statements  and  new  statements.  An  incumbent  who  took  Form  80  or 
Form  90  of  an  invenrory  received  credit  for  a  recovered  statement  when  he  wrote  in  one  of  the 
statements  intentionally  omitted  from  these  forms.  Credit  was  given  for  a  new  statement  when 
a  write-in  was  judged  equivalent  to  one  of  the  new  statements  on  the  scoring  form.  If  the 
rates  at  which  recovered  and  new  statements  were  written  in  by  incumbents  had  proved  equiv¬ 
alent,  the  results  from  both  types  of  statements  could  have  been  pooled  without  analyzing  the 
data  of  each  type  separately.  Since  preliminary  analyses  s'  ggested  different  rates  of  recovery, 
three  dependent  variables  were  calculated  to  describe  yields  of  information.  One  of  the  depen¬ 
dent  variables,  Xj,  was  based  on  both  new  and  recovered  statements;  a  second  variable,  X2, 
was  based  only  on  new  statements;  and  the  third  variable,  Xj0,  was  derived  solely  from  the 
intentionally  omitted  statements. 

Xj,  X2.  and  X50  may  all  be  interpreted  as  proportions.  Xj  is  the  ratio  of  the  length  of 
an  inventory  obtained  from  a  particular  subgroup  to  the  length  of  the  inventory  obtained  from 
using  all  respondenrs.  Thus: 


*1 


No.  of  statements  in  No.  of  statements  No.  of  new  statements 

form  the  group  took  +  recovered  by  group  +  obtained  from  group 

No.  of  statements  Total  of  new 
in  Form  100  +  statements 


Example:  A  subgroup  of  20  Ground  Radio  Operators  took  Form  80,  recovered  7  statements, 
and  yielded  69  new  starements. 

x  =  67  +  7+69  =  143 
1  84  +  186  270 

Variable  X2  was  based  only  on  new  statements  and  was  determined  for  a  group  by  divid¬ 
ing  the  number  of  different  new  statements  obtained  from  the  group  by  the  total  number  of  dif¬ 
ferent  new  statements  obtained  from  all  360  men  in  the  specialty.  When  X2  was  calculated  for 
the  first  sample  of  20  Ground  Radio  Operators  who  took  Form  80,  it  equalled  69  divided  by 
186  or  .371.  The  number  of  new  statements  written  in  by  rhe  group  was  69  and  the  number  of 
new  statements  written  in  by  all  360  men  was  186. 

Variable  Xj0  was  based  on  the  intentionally  omitted  statements  only.  Since  no  statements 
were  intentionally  omitted  from  Form  100,  X50  was  not  defined  for  groups  which  took  this  form. 
The  numerator  of  X50  for  a  group  was  the  number  of  differenr  recovered  statements  produced 
by- the  group.  The  denominator  was  the  number  of  starements  which  the  group  might  have  re¬ 
covered  (the  number  of  statements  intentionally  omitted  from  the  form  in  question)  .  When 
calculated  for  die  same  sample  of  Ground  Radio  Operators  used  in  the  above  examples,  *50 
equalled  7/17  or  .412.  The  group  recovered  7  of  the  17  statements  intentionally  omirted  from 
the  form. 
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The  maximum  possible  value  for  X|,  and  was  1 .  The  minimum  possible  value 
for  A' 2  and  .V50  was  0.  For  A,,  the  minimum  possible  value  was  a  variable  depending  on  the 
length  of  the  original  inventory  and  the  length  of  the  scoring  form.  Even  if  the  incumbents 
within  a  group  provided  no  write-ins,  Aj  would  not  be  0,  since  it  is  essentially  the  ratio  of 
the  length  of  a  revised  inventory  derived  from  a  single  group  to  the  length  of  the  inventory 
derived  from  all  360  incumbents.  The  minimum  value  for  X,  for  each  form-specialty  combina¬ 
tion  is  indicated  in  Table  2. 

The  data  used  in  the  multiple  regression  analyses  performed  on  variables  X,,  X2.  and 
*50  afc  presented  in  Tables  2,  3,  and  4.  In  Table  2  values  of  the  Xj  criterion  appear  as  a 
function  of  sample  size  for  each  combination  of  specialty  and  form.  Ranges  of  values  were 
.41— .65  for  Ground  Radio  Operators,  .59— .77  for  Automotive  Repairmen,  .58— .80  for  Aircraft 
Hydraulic  Repairmen,  and  .40-. 69  for  Accounting  Sc  Finance  Specialists. 

Perhaps  the  most  immediately  striking  aspect  of  these  data  is  that  the  values  are  not 
extremely  high.  Even  when  a  sample  of  size  60  was  selected,  in  no  case  did  the  value  exceed 
.80.  The  point  of  dimini.  >.ing  returns  was  not  reached  with  samples  of  size  60.  Though  it 
might  have  been  anticipated  that  60  specialists  would  provide  sufficient  information  to  obtain 
a  95-percent  complete  inventory,  80  percent  was  the  maximum  they  provided.  Increases  in 
sample  sizes  beyond  60  would  have  yielded  a  considerable  number  of  additional  unique  state¬ 
ments.  These  findings  indicate  that  there  must  be  statements  which  have  very  low  probabilities 
of  recovery,  since  they  are  written  in  rarely.  If  all  individual  statements  had  high  probabilities 
of  recovery,  small  samples  of  incumbents  would  have  been  sufficient  for  high  yields  of  informa¬ 
tion.  At  a  later  point,  the  distribution  of  "probability  of  recovery"  for  individual  statements 
is  examined. 

Some  of  the  same  general  comments  which  applied  to  Xj  are  appropriately  made  with 
respect  to  X2  and  X5 0  (Tables  3  and  4).  Samples  of  size  60  were  not  sufficient  to  produce 
very  high  yields  on  either  measure.  One  value  of  X2  was  only  .09,  and  the  highest  value,  one 
obtained  with  a  sample  of  60  Aircraft  Hydraulic  Repairmen,  was  only  .64.  The  range  of  X50 
was  from  .05  to  .78. 

At  least  two  sources  of  evidence  point  to  a  difference  between  new  and  recovered  state¬ 
ments.  In  the  72  samples  for  which  both  measures  are  defined,  X50  is  greater  than  X2  in  18 
out  of  18  times  for  Ground  Radio  Operators,  9  out  of  18  times  for  Accounting  Sc  Finance  Spe¬ 
cialists,  16  out  of  18  times  for  Automotive  Repairmen,  and  6  out  of  18  times  for  Aircraft 
Hydraulic  Repairman.  These  two  measures  correlated  only  .51. 

Table  5  lists  the  16  variables  used  in  the  group  analyses.  The  first  three  variables, 

Xj.  X2,  and  X5Q,  have  already  been  described.  For  purposes  of  these  analyses,  AFSC  be¬ 
came  fout  variables;  Form  became  three  variables.  Variable  Xjq  is  continuous  sample  size, 
and  Variable  Xu  is  the  square  of  continuous  sample  size.  Three  variables  are  used  to  define 
categor.cal  size,  and  the  last  variable  is  the  number  of  items  in  the  100-percent  form  of  the 
inventory  given  to  the  group  in  question.  The  square  of  continuous  sample  size  was  included 
as  a  predictor  variable  in  order  that  the  regression  equation  used  in  predicting  yield  as  a 
function  of  sample  size  might  be  quadratic  in  form.  It  seemed  reasonable  that  yield  should  be 
a  negatively  accelerated  positive  function  of  sample  size.  The  square  of  continuous  sample 
size  was  included  as  a  predictor  variable  to  fit  the  data  with  part  of  a  parabola. 

The  intercorrelations  of  the  16  variables,  based  on  all  108  observations,  were  computed. 
Table  6  contains  the  results  of  the  multiple  correlations  obtained  for  the  group  analyses.  The 
multiple  correlation  between  X,  and  all  predictors  from  X}  to  Xu  was  .935.  As  might  be 
anticipated,  a  high  multiple  correlation  of  .753  was  obtained  between  AFSCs  and  X^  Part  of 
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(Text  continues  on  p.  22) 


Table  2.  Regression  Equation,  Obtained  Sample,  and  Minimum  Values  of  X, 


Value 

Source 

Ground  Radio 
Operator 
Samples 

Automotive 

Repairman 

Samples 

Aircraft  Hy¬ 
draulic  Repair¬ 
man  Samples 

Accounting  At 
Finance  Spe- 
cialiat  Samples 

Foim 

20 

40 

60 

20 

40  60 

20 

40 

60 

20 

40  60 

80- 

pcrccnt 

Equation 
Replication  1 
Replication  2 
Replication  3 
Minimum 

.46 

.53 

.53 

.46 

.25 

.53 

.62 

.53 

.52 

.59 

.63 

.62 

.61 

.61 

.63 

.62 

.59 

.51 

.68  .74 
.70  .72 

.67  .71 

.66  .68 

.60 

.63 

.62 

.58 

.41 

.67 

.68 

.65 

.63 

.73 

.77 

.76 

.74 

.47 

.43 

.41 

.40 

.26 

.54  .61 
.57  .65 
.57  .61 

.50  .58 

90- 

percent 

Equation 
Replication  1 
Replication  2 
Replication  3 
Minimum 

.48 

.42 

.41 

.41 

.28 

.55 

.58 

.57 

.46 

.61 

.60 

.57 

.56 

.62 

.67 

.66 

.63 

.57 

.70  .76 

.73  .77 

.69  .76 

.66  .73 

.61 

.65 

.65 

.64 

.46 

.69 

.70 

.70 

.69 

.75 

.79 

.75 

.75 

.49 

.52 

.47 

.43 

.29 

.56  .62 
.63  .69 
.59  .68 
.59  .66 

100- 

percent 

Equation 
Replication  1 
Replication  2 
Replication  3 
Minimum 

.48 

.51 

.45 

.45 

.31 

.55 

.54 

.53 

.52 

.61 

.65 

.59 

.57 

.62 

.68 

.68 

.67 

.62 

.70  .76 
.72  .72 

.69  .72 
.69  .71 

.61 

.67 

.62 

.60 

.51 

.69 

.74 

.74 

.71 

.75 

.80 

.79 

.78 

.49 

.48 

.47 

.47 

.32 

.56  .62 
.59  .62 
.56  .60 
.55  .59 

Table  3.  Regression  Equation  and  Obtained  Sample  Values  of  the  X2  Criterion 


Form 


Value 

Source 


Ground  Radio 
Operator 
Samples 


Automotive 

Repairman 

Samples 


Aircraft  Hy¬ 
draulic  Repair¬ 
man  Samples 


Accounting  & 
Finance  Spe¬ 
cialist  Samples 


80-  Equation 

percent  Replication  1 
Replication  2 
Replication  3 

90-  Equation 

percent  Replication  1 
Replication  2 
Replication  3 

100-  Equation 
percent  Replication  1 
Replication  2 
'  Replication  3 


20 

40 

60 

20 

40 

60 

20 

40 

60 

20 

40 

.25 

.37 

.47 

.15 

.28 

.38 

.35 

.47 

.57 

.27 

.40 

.37 

.49 

.51 

.21 

.34 

.41 

.37 

.46 

.63 

.23 

.40 

.37 

.37 

.49 

.18 

.30 

.36 

.34 

.41 

.62 

.21 

.40 

.27 

.35 

.47 

.15 

.27 

.34 

.26 

.38 

.55 

.19 

.31 

.25 

.37 

.47 

.15 

.28 

38 

.35 

.47 

.57 

.27 

.40 

.19 

.40 

.43 

.20 

.36 

.46 

.36 

.47 

.64 

.33 

.48 

.18 

.39 

.39 

.20 

.27 

.45 

.36 

.45 

.56 

.26 

.42 

.18 

.25 

.38 

.16 

.20 

.38 

.34 

.44 

.54 

.21 

.42 

.19 

.32 

.42 

.10 

.22 

.32 

.29 

.41 

.51 

.22 

.34 

.29 

.34 

.49 

.13 

.23 

.25 

.33 

.48 

.59 

.23 

.39 

.20 

.32 

.41 

.12 

.17 

.23 

.22 

.47 

.57 

.22 

.35 

.20 

.31 

.38 

.09 

.15 

.21 

.18 

.40 

.56 

.21 

.33 

60 


.57 


.44 


Table  4.  Regression  Equation  and  Obtained  Sample 

Values  of  the  X50 

Criterion 

Ground  Radio 
Operator 
Samples 

Automotive 

Repairman 

Samples 

Aircraft  Hy¬ 
draulic  Repair¬ 
man  Samples 

Accounting  & 
Finance  Spe¬ 
cialist  Samples 

Value 

Form  Source 

20  40  60 

20  40  60 

20  40  60 

20  40  60 

80-  Equ  ation 

percent  Replication  1 
Replication  2 
Replication  3 

90-  Equation 

percent  Replication  1 
Replication  2 
Replication  3 


.42 

.57 

.60 

.32 

.46 

.50 

.36 

.51 

.54 

.33 

.47 

.51 

.41 

.53 

.53 

.47 

.59 

.55 

.49 

.51 

.59 

.37 

.66 

.63 

.41 

-53 

.53 

.30 

.50 

.47 

.43 

.47 

.57 

.29 

.55 

.58 

.35 

.41 

.53 

.24 

.46 

.37 

.41 

.36 

.50 

.24 

.45 

.45 

.34 

.49 

.52 

.23 

.38 

.41 

.28 

.42 

.46 

.24 

.39 

.42 

.33 

.78 

.67 

.36 

.40 

.47  • 

.33 

.36 

.53 

.32 

.47 

.42 

.33 

.67 

.67 

.26 

.28 

.45 

.31 

.36 

.53 

.16 

.37 

.42 

.33 

.33 

.56 

.08 

.26 

.40 

.31 

.36 

.39 

.05 

.32 

•  32 

Note.— Xmi  *a  not  defined  for  groups  which  took  Form  100,  since  the  denominstor  of 

•  _  r _ _ _  —  f mm  ■  frtim. 
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X-n  is  the  number  of  tssk  statements  omitted  from  a  toim. 
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Table  5.  Variables  of  the  Group  Analyses 


Variable 

Name 

Variable 

Name 

*1 

New  and  Recovered  Statements 

*8 

Form  90 

*2 

New  Statements 

x9 

Form  100 

•So 

Recovered  Statements 

*10 

Sample  Size 

*3 

Ground  Radio  Operator 

*11 

Square  of  (X10) 

*4 

Accounting  &  Finance  Specialist 

*12 

Sample  Size  20 

*5 

Automotive  Repairman 

*13 

Sample  Size  40 

*6 

Aircraft  Hydraulic  Repairman 

*14 

Sample  Size  60 

*7 

Form  80 

*51 

Number  of  Items  in  Form  100 

Table  6.  Multiple  Correlations  From  Group  Analyses 
(N  =  108) 


Variables 

Included 

Content  Included 

*1 

New  &  Recovered 
Statements 

*2 

New 

Statements 

*50 

Recovered 

Statements" 

*3  ~  *14 

AFSCrForm,  Continuous- Sample 
Size,  Categorical  Sample  Size 

.935 

.911 

.729 

*3- *6 

AFSC 

.753 

.526 

.324 

*7- *9 

Form 

.086b 

.217b 

.305 

*12“  *14 

Categorical  Sample  Size 

.549 

.711 

.577 

*10~  *11 

Variables 

Deleted 

Continuous  Sample  Size 

Content  Deleted 

.549 

.711 

.577 

*3  ~  *6-  *51 

AFSC,  No.  Items  in  Form  100 

.722 

.846 

.653 

*7-  *9-  *51 

Form,  No.  Items  in  Form  100 

.931 

.885 

.662 

*10“  *14’  *51 

Continuous  Sample  Size,  Cate¬ 
gorical  Sample  Size,  No. 

Items  in  Form  100 

.757 

.5  69 

.445 

*13“  *14-  *51 

Categorical  Sample  Size, 

No.  Items  in  Form  100 

.935 

.822 

.433 

Correlation  with  X51 

*  Based  on  Forms  80  and  90  only,  N  =  72. 
bNot  significant  at  .05  level. 

.698 

-.  1 83  b 

-.225b 
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this  correlation  may  be  attributed  to  the  fact  that  the  minimum  value  of  A'j  is  different  for 
diffetent  AFSCs.  The  correlation  between  Form  and  X,  was  not  significant.  The  correlation 
of  continuous  sample  size  with  Xj  turned  out  to  be  reasonably  large  (.549)  and  exactly  equal 
to  the  correlation  of  categorical  sample  size  with  Xj.  In  fact,  the  two  sets  of  sample-size 
variables  always  correlated  identically  with  each  of  the  criterion  variables.  This  is  reason¬ 
able  as  sample  size  squared  is  included  in  the  combination  of  variables  called  continuous 
sample  size.  Since  the  variable  of  sample  size  has  only  three  values,  a  quadratic  equation 
can  define  the  regression  line  as  adequately  as  any  combination  of  the  categorical  variables, 
X12,  Xjj.  and  X14.  It  was  observed,  however,  in  the  correlation  matrix  that  the  correlations 
of  X10  with  the  criterion  variables  are  only  slightly  less  than  the  correlations  when  X10  and 
Xjj  are  joint  predictors. 

The  lower  part  of  Table  6  shows  the  multiple  correlations  obtained  when  some  sets  of 
variables  were  omitted  or  deleted.  Omitting  AFSC  had  the  greatest  effect  on  the  multiple 
correlation.  Omission  of  the  sample-size  variables  had  the  next  greatest  effect  and  resulted 
in  a  correlation  of  .757  with  Xj. 

Though  the  general  pattern  of  the  correlations  of  the  predictor  variables  with  X2  and  XJ0 
was  similar  to  that  obtained  with  Xj,  there  were  a  few  differences  of  note.  With  minor  excep¬ 
tions,  the  correlations  of  the  predictors  with  X,0  were  lower  than  the  comparable  correlations 
with  X2.  The  highest  multiple  correlation  fot  X50  was  .729.  The  independent  contribution  of 
AFSC  was  smaller  for  X2  and  X,0  than  for  Xj.  Undoubtedly  this  result  was  associated  with 
the  fact  that  the  minimum  values  of  X2  and  Xjq  were  not  functions  of  AFSC.  Sample  size 
remained  an  important  variable  as  is  shown  by  the  marked  reductions  in  the  multiple  correlation 
when  the  sample-size  variables  were  deleted. 

The  regression  equation  obtained  from  variables  Xj  through  X 14  to  predict  Xj  was  used 
as  a  basis  for  obtaining  the  predicted  values  of  Xj  which  are  entered  in  Table  2.  Comparable 
equations  were  used  to  predict  values  of  X2  and  Xjq.  Examination  of  Tables  2,  3,  and  4  gives 
some  idea  of  the  adequacy  of  the  fit  of  the  regression  lines  to  the  obtained  data. 

It  might  be  questioned  whether  the  obtained  regressions  of  the  yield  variables  on  sample 
size  can  legitimately  be  extrapolated  beyond  sample  size  60.  Extrapolation  is  rarely  possible 
without  some  qualification,  and  this  case  is  no  exception.  The  data  for  samples  of  size  0  and 
360  were  not  included  in  the  analyses,  but  values  of  Xj  (and  X2  and  X50)  can  be  determined 
for  samples  of  these  sizes.  With  samples  of  size  zero,  Xj  equals  its  minimum  value;  X2  and 
XJ0  equal  0.  For  the  samples  of  360,  upon  which  the  scoring  forms  were  based,  Xj  would  be 
very  close  to  1.00  and  X2  and  *50  would  equal  1.00. 

Extrapolations  of  the  regression  equations  downward  in  the  direction  of  a  sample  of  size 
of  zero  produce  some  fairly  sizable  overestimates  of  the  minimum  values  of  the  criterion  vari¬ 
able;  extrapolations  upward  in  the  direction  of  samples  of  size  360  produce  some  not  very 
reasonable  predictions.  As  sample  size  increases,  the  predicted  values  eventually  reach  a 
maximum  and  then  become  a  decreasing  function  of  sample  size.  This  is  understandable  if  it 
is  recalled  that  the  regression  equations  of  yield  on  sample  size  are  parabolas.  If  the  purpose 
of  the  analysis  had  been  to  extrapolate  beyond  the  particular  sample  sizes  employed,  some 
other  kind  of  function  would  have  been  preferable.  Simple  linear  functions  are  one  possibility 
and  these  might  have  fitted  the  data  reasonably  well,  since  the  functions  are  very  close  to 
linear  over  the  range  of  sample  sizes  selected. 

In  some  respects  the  results  of  the  group  analyses  were  quite  unexpected.  Most  un¬ 
expected  was  die  way  in  which  yield  of  information  continued  to  increase  as  sample  size 
increased.  This  finding  is  expected  when  there  are  many  statements  which  have  extremely 
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low  probabilities.  If  a  statement  is  written  in  only  once  or  twice  in  a  total  sample  of  360 
incumbents,  it  is  apparent  that  a  small  sample  might  not  include  the  individual  or  individuals 
who  wrote  in  the  statement.  Table  7  presents  information  relative  to  this  point.  It  shows 
that,  for  the  totals  across  specialties,  one-fourth  of  the  incumbents  produced  no  new  state¬ 
ments,  and  over  a  half  no  more  than  three  statements.  Table  8  shows  the  relative  frequency 
of  write-ins  of  the  same  statement.  Over  30  percent  of  the  new  statements  are  offered  by  just 
one  man  out  of  the  360.  The  chances  of  getting  a  specific  one  of  those  275  statements  from 
a  random  sample  of  40  incumbents  are  small.  The  considerable  difference  in  productivity  for 
men  in  the  four  specialties  appears  in  both  Tables  7  and  8.  Ground  Radio  Operators  and  Auto¬ 
motive  Repairmen  have  a  lower  frequency  of  write-ins  and  less  duplication  of  new  statements. 
"Popular”  statements  were  most  common  in  the  Hydraulic  specialty  with  about  13  percent  of 
the  statements  appearing  20  or  more  times.  For  the  Automotive  Repairman  and  Accounting  & 
Finance  Specialties,  the  comparable  figures  were  less  than  5  percent.  No  statement  was  ever 
written  in  by  more  than  one-third  of  the  men  of  any  specialty. 

Table  7.  Distribution  of  Number  of  New  Task  Statements 
One  Incumbent  Produces 


Number  of  Incumbents  Who  Wrote  in  n  New  Statements 


n:  number  of 
new  statements 
written  in 

Ground  Radio 
Operator 

Automotive 

Repairman 

Aircraft 

Hydraulic 

Repairman 

Accounting  & 
Finance 
Specialist 

Total 

0 

90 

140 

81 

49 

360 

1 

57 

59 

23 

38 

177 

2 

35 

40 

24 

40 

139 

3 

36 

24 

25 

30 

115 

4 

31 

25 

15 

28 

99 

5 

26 

14 

18 

23 

81 

6 

22 

14 

14 

20 

70 

7 

16 

8 

19 

19 

62 

8 

11 

13 

13 

21 

58 

9 

14 

2 

10 

15 

41 

10 

4 

5 

14 

11 

34 

11 

3 

2 

9 

99 

23 

12 

3 

3 

8 

9 

23 

13 

2 

2 

5 

3 

12 

14 

2 

2 

7 

7 

18 

15 

1 

1 

10 

1 

13 

16 

3 

0 

4 

13 

20 

17 

1 

3 

5 

4 

13 

18 

0 

0 

3 

4 

7 

19 

0 

0 

3 

2 

5 

20 

0 

0 

1 

2 

3 

More  than  20 

3 

3 

49 

12 

67 

Sample  Size  Prediction 

When  the  regression  equations  were  derived  from  the  group  analysis,  yield  was  predicted 
from  AFSC  and  sample  size.  Instead  of  predicting  yield,  it  is  possible  to  set  the  value  of 
yield  and  to  solve  the  equation  "backwards”  for  sample  size.  A  computer  program  was  written 
and  numerical  solutions  were  obtained  for  the  case  in  which  Xj  was  the  criterion  variable,  and 
AFSC,  Form,  Sample  Size,  and  the  Square  of  Sample  Size  were  the  predictor  variables.  A  series 
of  criterion  yield  values  (Xj)  ranging  from  .45  to  .95  in  steps  of  .05  were  inserted  in  the  quad¬ 
ratic  equations  for  each  of  the  12  combinations  of  specialties  and  forms.  Then  the  quadratic 
equations  were  solved  to  give  the  predicted  sample  size. 
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Table  9  shows  estimated  sample  sizes  for  specified  yields  of  information  on  variable 
In  each  case,  the  values  of  Form  90  and  Form  100  are  the  same  since  the  Form  90  and  100 
variables  (Xg  and  A'9)  were  not  selected  in  the  regression  systems.  In  all  specialties,  the 
estimated  sample  sizes  for  Form  80  are  somewhat  higher  than  those  for  Forms  90  and  100. 


Table  8.  Percentage  of  New  Statements 
Written  in  a  Given  Number  of  Times 


n:  number  of 
new  statements 
written  in 

Number  of  Incumbents  Who  Wrote  in 

n  New  Statements 

Ground  Radio 
Operator 

Automotive 

Repairman 

Aircraft 

Hydraulic 

Repairman 

Accounting  & 
Finance 
Specialist 

Total 

1 

36.56 

48.01 

22.46 

24.16 

31.4 

2 

17.74 

19.13 

13.47 

19.94 

17.6 

3 

7.53 

8.30 

9.58 

11.24 

9.5 

4 

8.06 

5.05 

7.19 

7.30 

6.9 

5 

2.69 

6.50 

5.99 

7.30 

6.0 

6 

2.69 

1.81 

1.50 

4.49 

2.8 

7 

1.61 

.36 

2.99 

3.37 

2.2 

8 

2.15 

1.81 

3.59 

1.69 

2.3 

9 

4.30 

.36 

2.69 

2.81 

2.4 

10 

.54 

1.81 

2.99 

1.40 

1.8 

11 

.54 

.72 

1.50 

1.97 

1.3 

12 

1.08 

1.08 

1.20 

2.25 

1.5 

13 

2.69 

1.08 

3.59 

.56 

1.9 

14 

.54 

.00 

.60 

.84 

0.5 

15 

.00 

.36 

2.10 

.84 

0.9 

16 

.00 

.36 

1.80 

1.69 

1.1 

17 

.54 

.00 

1.20 

1.12 

0.8 

18 

.54 

.00 

.60 

.56 

0.4 

19 

.54 

.72 

.60 

1.40 

0.9 

20 

.54 

.00 

1.80 

.84 

0.9 

More  than  20 

9.17 

2.52 

12.60 

4.20 

7.0 

Table  9.  Estimated  Sample  Sizes  for  Specified  Yields 


X, 

Subgroup 

.45 

.50 

.55 

.60 

.65 

.70 

.75 

.80 

.85 

.90  .95 

Ground  Radio  Operator 

46.1 

63.0 

83.5 

112.0 

—Form  80 

18.2 

31.4 

—Form  90 

13.7 

26.5 

40.5 

56.5 

75.4 

100.0 

151.9 

—Form  100 

13.7 

26.5 

40.5 

56.5 

75.4 

100.0 

151.9 

Accounting  &  Finance  Specialist 
—Form  80 

14.7 

27.5 

41.7 

57.9 

77.1 

102.4 

—Form  90 

10.3 

22.7 

36.3 

51.7 

69.6 

92.1 

127.3 

—Form  100 

10.3 

22.7 

36.3 

51.7 

69.6 

92.1 

127.3 

Automotive  Repairman 

-15.3 

-4.8 

6.3 

18.4 

31.6 

46.3 

63.2 

83.8 

112.5 

155.7 

-19.0 

-8.7 

2.2 

13.9 

26.6 

40.7 

56.7 

75.7 

100.4 

—Form  100 

-19.0 

-8.7 

2.2 

13.9 

26.6 

40.7 

56.7 

75.7 

100.4 

155.7 

Aircraft  Hydraulic  Repairman 

-17.4 

-7.1 

3.9 

15.8 

28.7 

43.1 

59.5 

79.1 

105.3 

132.7 

-21.0 

-10.9 

-.2 

11.4 

23.9 

37.7 

53.2 

71.4 

94.5 

—Form  100 

-21.0 

-10.9 

-.2 

11.4 

23.9 

37.7 

53.2 

71.4 

94.5 

132.7 

24 


The  differences,  however,  are  relatively  small  and  there  is  no  simple  explanation  because  of 
the  lack  of  differences  between  Forms  90  and  100.  However,  if  the  proportion  of  the  omitted 
statements  recovered  were  a  constant  from  form  to  form,  one  would  expect  differences  in  esti¬ 
mated  yield  of  the  type  observed  in  comparing  Form  80  with  Forms  90  and  100. 

The  solutions  of  the  quadratic  equations  indicate  the  necessity  of  sample  sizes  of  more 
than  100  for  yields  of  .75  in  the  Ground  Radio  Operator  and  Accounting  Sc  Finance  Specialties. 
Though  the  situation  is  somewhat  better  for  the  Automotive  Repairman  and  Aircraft  Hydraulic 
Repairman  specialties,  the  estimated  sample  sizes  for  a  yield  of  .85  are  all  close  to  100. 

In  interpreting  these  estimates,  it  should  be  realized  that  they  are  based  on  extrapolations 
of  the  regression  equation  beyond  sample  sizes  for  which  data  were  collected.  The  largest 
samples  actually  drawn  were  of  size  60.  Sample  size  estimates  larger  than  60  should  be  inter¬ 
preted  with  caution  because  extrapolation  becomes  more  tenuous  as  estimated  sample  size 
increases  beyond  this  point.  The  necessity  for  caution  is  further  emphasized  by  the  absence  of 
estimates  for  several  high  yield  values  in  Table  9.  This  is  because  the  parabola  defining  the 
regression  of  yield  on  sample  size  becomes  a  decreasing  function  somewhere  between  the  last 
yield  for  which  a  tabled  sample  size  is  given  and  the  immediately  higher  yield.  When  the  quad¬ 
ratic  equations  are  solved  for  sample  size,  the  quantity  under  the  radical  is  negative  for  yields 
beyond  the  point  at  which  the  quadratic  becomes  a  decreasing  function. 

The  negative  numbers  in  Table  9  occur  because  the  yield  values  which  have  been  sub¬ 
stituted  in  the  equation  are  less  than  the  Xj  values  the  equation  would  prediqt  for  a  sample 
size  of  zero. 

Individual  Analyses 

The  purpose  of  the  individual  analyses  was  to  determine  the  extent  to  which  certain 
characteristics  of  the  individual,  such  as  his  specialty,  the  completeness  of  the  form  admin¬ 
istered  to  him,  the  command  to  which  he  was  assigned,  the  number  of  months  he  had  spent  in 


Table  10.  Variables  of  the  Individual  Analyses 


Variable 

Name 

Variable 

Name 

*15 

Quantity  Index 

*27 

Months  in  AFSC  at  5-level 

*16 

Quality  Index 

*28 

Education  in  years 

*17 

Ground  Radio  Operaror 

*29 

Age 

*18 

Accounting  &  Finance  Specialist 

*30 

Rank 

*19 

Automotive  Repairman 

*31 

Definitely  a  career  airman 

*20 

Aircraft  Hydraulic  Repairman 

*32 

APT  score 

*21 

Form  80 

*33 

Mechanical  AI 

*22 

Form  90 

*34 

General  or  Technical  AI 

*23 

Form  100 

*35 

Clerical  AI 

*24 

SAC 

*36 

Electronics  AI 

*25 

TAC 

*37 

Time  to  complete  inventory 

‘’lb 

ATC 

*38 

Months  OJT  at  5-level 
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his  specialty  at  the  five-level*  and  a  number  of  background  variables  such  as  his  education, 
age,  rank,  career  intentions,  aptitude  indexes,  and  achievement  scores,  were  related  to  the 
quantity  and  quality  of  information  which  he  yielded.  Table  10  lists  the  variables  used  in  the 
individual  analyses.  There  were  two  dependent  variables  in  these  analyses.  The  first,  X15. 
was  a  quantity  index  which  was  based  solely  on  write-ins  of  new  statements.  For  a  particular 
individual,  Xj  5  was  defined  as  the  ratio  of  the  number  of  new  statements  he  wrote  to  the  total 
number  of  different  statements  written  in  by  the  entire  sample  for  his  specialty. 

A  second  variable,  X16,  provided  an  index  of  the  quality  of  an  incumbent’s  write-ins. 

The  basic  notion  underlying  the  use  of  \16  was  that  an  incumbent  who  writes  in  rare  state¬ 
ments,  ones  not  likely  to  be  provided  by  other  men, is  potentially  a  good  candidate  for  inclusion 
in  a  sample,  assuming  he  can  be  identified  by  suitable  predictor  variables.  The  definition  of 
\  was  revised  on  the  basis  of  data  gathered  from  the  Ground  Radio  Operator  specialty.  As 
originally  defined,  ,V16  was  so  highly  correlated  with  X15  that  it  yielded  practically  no  new 
information.  The  reason  for  the  high  correlation  is  apparent  when  the  data  and  the  nature  of 
the  originally  proposed  rarity  measure  are  considered.  Under  the  original  definition,  the  in¬ 
cumbent’s  credit  was  inversely  proportional  to  the  probability  of  the  write-in.  If  an  incumbent 
wrote  in  a  statement  which  had  been  written  in  by  .05  of  all  of  the  men  within  the  specialty,  he 
was  credited  with  a  value  of  1.00  minus  .05,  or  .95,  for  that  statement.  A  similar  value  was 
computed  for  all  new  statements  which  he  provided.  These  were  added  to  form  the  numerator 
for  X16.  The  denominator  of  X16  was  based  upon  similar  calculations  for  all  the  new  statements 
he  might  have  provided;  in  other  words,  the  total  number  of  new  statements  obtained  from  all 
men  in  the  specialty.  It  is  easy  to  see  how  the  numerator  of  this  index  would  be  affected  if 
new  statements  rather  generally  had  low  probabilities  of  recovery  and  if  the  probabilities  of 
recovery  were  relatively  homogeneous.  In  the  limiting  case  of  identical  probabilities  of  recovery, 
incumbents  would  receive  equal  credit  for  a'l  statements.  With  nearly  equal  and  low  probabili¬ 
ties  of  recovery,  the  numerator  becomes  practically  identical  to  number  of  new  statements  written 
in  by  the  incumbents.  Since  individual  statements  were  in  general  characterized  by  low  and 
homogeneous  probabilities  of  recovery,  this  result  was  obtained. 

Several  alternative  rarity  scales  were  tried  out  with  a  view  to  distinguishing  more  sharply 
among  providers  of  rare  information.  Since  few  of  the  statements  obtained  from  any  specialty 
were  ’’very  common,”  and  whereas  many  were  ’’very  rare,”  it  seemed  appropriate  to  construct 
a  scale  such  that  the  finest  distinctions  would  be  drawn  among  the  extremely  rare  statements. 
The  measure  selected  was  based  on  a  5-point  scale  which  gave  the  highest  rarity  rating  to  a 
small  group  of  ’’most  rare”  statements,  and  progressively  lower  ratings  to  progressively  larger 
groups  of  "less  rare”  statements.  ' 

The  calculation  of  the  newly  defined  X16,  the  one  eventually  used  with  all  specialties, 
may  be  illustrated  by  a  consideration  of  the  Ground  Radio  Operator  specialty.  All  of  the  186 
different  new  task  statements  obtained  were  listed  in  a  frequency  distribution  in  the  order  of 
the  number  of  times  each  was  written  in.  The  sum  of  the  frequencies  for  these  statements  was 
1269.  The  statements  were  then  divided  into  five  groups  of  unequal  size,  the  largest  group 
(5/15  x  1269)  containing  the  most  common  statements,  and  progressively  smaller  groups  (4/15 
x  1269  ,  3/15  x  1269  ,  2/15  x  1269,  1/15  x  1269)  containing  statements  of  progressively 
greater  rarity.  The  most  rare  statements  were  given  a  value  of  5,  the  least  rare  a  value  of  1. 
Subsequent  to  obtaining  the  rating  values  for  each  statement,  the  rarity  rating  for  an  incumbent 
was  determined  by  calculating  the  mean  of  the  rating  values  for  the  statements  which  he  wrote 


The  definition  of  X16  removes  to  a  considerable  extent  the  direct  effect  of  number  of 
write-ins  on  the  value  of  the  measure.  In  fact,  a  man  who  wrote  in  a  larger  number  of  statements 
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including  a  few  rare  statements  could  get  a  "poorer”  rarity  score  than  another  individual  who 
wrote  in  but  a  single  very  rare  statement.  In  this  sense,  the  rarity  index  is  not  a  pure  measure 
of  quality.  Such  a  measure  probably  exists  somewhere  between  the  limits  defined  by  a  quantity 
and  a  rarity  measure. 

Information  on  the  background  variables  was  obtained  at  the  time  of  the  initial  adminis¬ 
tration  of  the  inventories.  Each  incumbent  completed  an  identification  sheet  giving  his  com¬ 
mand,  the  number  of  months  he  had  been  at  the  five-level, his  age,  educational  level,  rank,  etc. 
The  test  control  officer  filled  out  an  information  sheet  on  each  examinee  which  supplied  scores 
for  Airman  Proficiency  Tests  (APTs)  and  Aptitude  Indexes  (AIs).  Since  the  records  from  which 
this  information  was  obtained  were  not  always  complete,  values  of  some  of  the  variables  were 
estimated  for  some  of  the  incumbents.  Approximately  15  percent  of  the  scores  had  to  be  esti¬ 
mated  on  the  APT  and  AI  variables  to  obtain  complete  data  for  intercorrelation.  The  estimation 
was  accomplished  by  finding  another  individual  who  matched  the  individual  for  whom  the  predic¬ 
tion  had  to  be  made  as  closely  as  possible  on  the  other  predictor  variables.  The  corresponding 
scores  of  the  matched  individual  were  then  inserted  as  a  best  estimate  for  the  missing  values. 

Table  11  summarizes  the  multiple  correlations  of  the  predictors  with  X15  and  Xlfi.  The 
multiple  correlation  of  all  the  variables  with  Xl5  was  .464,  *hich  indicated  only  a  moderate 
degree  of  relationship  between  the  predictor  variables  and  the  quantity  criterion.  The  greatest 
single  contribution  to  overall  correlation  came  from  the  AFSC  variable.  When  AFSC  was  deleted, 
the  multiple  correlation  was  reduced  to  .329,  indicating  that  its  contribution  was  equal  to  about 
half  of  the  criterion  variance  accounted  for  by  all  predictors.  As  a  group,  the  "current  informa¬ 
tion”  variables,  which  included  AFSC,  form,  command,  and  months  in  the  AFSC,  make  the 
largest  contribution  to  the  criterion  variance.  Only  a  small  part  of  the  criterion  variance  is 
associated  with  the  remainder  of  the  variables. 

Multiple  correlations  with  X16  were  very  small.  Even  when  all  variables  were  used  as 
predictors,  the  multiple  correlation  was  only  .238.  No  single  predictor  variable  independently 
accounted  for  as  much  as  2  percent  of  the  criterion  variance. 

The  low  correlations  with  variables  X15  and  *16  indicate  that  the  predictors  analyzed 
would  not  be  very  helpful  in  selecting  individuals  who  are  likely  to  provide  high  yields  of 
information.  The  correlations  may  be  low,  in  part,  because  of  the  relative  homogeneity  of 
incumbents  on  the  criterion  variables.  Many  men  wrote  in  no  statements,  and  most  wrote  in 
no  more  than  a  few  (see  Table  7).  The  incumbent  who  wrote  in  a  large  number  of  acceptable 
statements  was  the  exception  rather  than  the  rule.  Perhaps  predictors  of  a  different  nature 
should  have  been  tried.  The  most  likely  variables  are  ones  related  to  the  potential  motivation 
of  the  incumbent  for  the  task.  Perhaps  the  greatest  number  of  write-ins  are  obtained  from  men 
who  like  to  fill  out  task  inventories. 

4.  SECOND  ADMINISTRATION 

A  complete  assessment  of  the  task  inventory  method  requires  information  not  only  con¬ 
cerning  the  number  of  tasks  yielded  by  incumbents,  but  also  concerning  the  nature  of  tasks 
which  are  performed  by  incumbents.  The  principal  purpose  of  this  part  of  the  research  was  to 
determine  the  interrelationship  of  a  number  of  task  variables.  Are  incumbents  more  likely  to 
write  in  a  statement  describing  a  missing  task  that  is  done  frequently,  that  takes  a  relatively 
long  time  to  do,  or  that  requires  more  than  average  training  and  experience?  If  there-is  a 
marked  relationship  between  the  likelihood  of  recovering  a  statement  and  some  of  the  variables 
describing  the  statement,  it  would  be  advisable  to  modify  the  task  inventory  to  increase  the 
likelihood  of  obtaining  tasks  drat  ate  done  infrequently,  take  little  time  to  perform,  or  require 
little  training  and  experience. 
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Table  II.  Multiple  Correlations  From  Individual  Analyses 

IN  -  1440) 


Variables 

Included 

Content  Included 

*15 

Quantity 

Criterion 

*16 

Quality 

Criterion 

*17  -  *20 

AFSC 

.409 

.144 

*21  _  *2  3 

Form 

.073“ 

.036* 

*2  4  ~  *26 

Command 

.026* 

.003* 

*21  ~  *27 

Form,  Command,  Months  in  AFSC 

.188 

.005* 

*28  "  *36 

Background  Variables 

.213 

.156 

*37  “  *38 

Time  to  Complete  Inventory,  Months  of  OJT 

.215 

.099 

*17  —  *38 

All  Variables 

.464 

.238 

Variables 

Deleted 

Content  Deleted 

*17  _  *20 

AFSC 

.329 

.213 

*21  ~  *23 

Form 

.460 

.234 

*2  4  ~  *26 

Command 

.464 

.234 

*27 

Months  in  AFSC 

.461 

.224 

*28 

Education  in  Years 

.463 

.236 

*29 

Age 

.461 

.238 

*30 

Rank 

.'464 

.226 

^31 

Career  Intention 

.462 

.236 

*32 

APT  Score 

.462 

.230 

*33 

Mechanical  AI 

.464 

.234 

*34 

General  or  Technical  AI 

.464 

.231 

*35 

Clerical  AI 

.464 

.221 

*36 

Electronics  AI 

.464 

.238 

*17  -  *27 

Current  Information  Variables 

.278 

.197 

*28  “  *36 

Background  Information  Variables 

.441 

.195 

*37 

Time  to  Complete  Inventory 

.448 

.207 

*38 

Months  of  OJT 

.464 

.238 

*37  “  *38 

Time  to  Complete  Inventory,  Months  of  OJT 

.447 

.207 

*Not  significant  at  .05  level. 
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Another  purpose  of  this  phase  was  to  develop  good  rating  scales  for  several  task  rati: 
factors  and  to  produce  revised  inventories  that  present  up-to-date  pictures  of  the  tasks  per¬ 
formed  in  four  Air  Force  specialties. 

The  materials  used  in  the  second  administration  were  the  revised  inventories  and  the 
task  rating  scales.  The  revised  inventories  differed  in  several  respects  from  their  earlier 
counterparts.  First,  they  were  much  longer.  The  Ground  Radio  Operator  inventory  increased 
from  84  statements  on  the  original  100-percent  form  to  270  statements  on  the  revised  form. 

The  inventories  for  Automotive  Repairman,  Aircraft  Hydraulic  Repairman,  and  Accounting  3c 
Finance  Specialist  were  increased  from  474  to  750  statements,  345  to  408  statements,  and 
177  to  546  statements  respectively.  The  revised  inventories  had  no  spaces  for  write-in  state¬ 
ments.  The  increase  in  the  length  of  the  inventories  and  the  addition  of  rating  scales  made 
it  impractical  to  solicit  write-ins.  In  fact,  the  original  form  in  which  the  revised  inventories 
were  prepared,  proved  too  long  to  administer  in  a  single  session  for  three  of  the  specialties. 

The  revised  inventories,  except  for  Ground  Radio  Operator,  were  divided  into  a  "First  Half" 
and  a  "Second  Half."  Different  men  took  different  halves.  Instructions  in  the  inventory  ex¬ 
plained  that  not  all  tasks  in  the  specialty  were  included  in  their  booklet,  and  an  outline  of  the 
missing  half  of  the  inventory  was  provided. 

Three  task  rating  scales  were  selected  for  administration  with  all  task  inventories. 

These  were  (1)  a  frequency  scale,  (2)  an  average-time  scale,  and  (3)  a  training  &  experience 
scale.  Several  preliminary  versions  of  the  scales  were  tried  out  prior  to  adoption  of  the  final 
scales  administered  with  the  revised  task  inventory.  The  early  versions  varied  both  in  content 
and  mode  of  administration.  For  example,  the  scale  that  finally  turned  out  to  be  concerned 
with  training  and  experience  was  initially  a  "difficulty”  scale.  The  term  "difficulty"  was  so 
variously  interpreted  (e.g.,  physical  difficulty,  monotony,  complexity)  that  it  was  not  suitable. 
The  principal  decisions  with  respect  to  the  time  and  frequency  scales  centered  not  so  much  on 
the  selection  of  the  scale  as  the  mode  of  asking  the  incumbents  to  report  the  time  and  frequency 
spent  on  tasks.  The  conclusion  from  the  preliminary  research  was  that  it  was  helpful  to  permit 
incumbents  to  choose  their  own  units  in  rating  tasks  on  time  and  frequency.  The  method  by 
which  this  was  done  and  the  time  and  frequency  scales  are  apparent  from  the  instructions  given 
to  incumbents.  These  instructions  are  reproduced  in  the  Appendix. 

Rating  scales  were  administered  in  booklets  separate  from  the  revised  inventory  booklets 
and  separate  from  each  other.  For  those  inventories  that  were  split  into  two  parts,  the  rating 
scale  booklets  were  split  similarly.  During  this  administration,  the  inventory  was  always  ad¬ 
ministered  first.  Incumbents  checked  "Yes”  next  to  those  statements  they  had  performed 
during  the  past  year.  The  three  rating  scale  booklets  were  administered,  in  turn,  after  the 
revised  inventory  had  been  completed. 

In  sending  out  the  inventories  to  the  test  control  officers  for  readministration,  a  list  of 
incumbents  at  each  base  who  had  taken  the  initial  form  was  included.  It  was  requested  that 
the  same  individuals,  where  possible,  be  given  the  readministration  form.  The  test  control 
officer  was  requested  to  administer  any  remaining  unused  inventories  and  rating  scales  to  five- 
level  airmen  in  the  appropriate  specialties  who  had  not  tak  n  the  initial  form.  It  turned  out 
that  only  about  half  of  the  returns  from  the  readministration  form  were  from  incumbents  who 
had  taken  the  initial  fotm. 

The  number  of  booklets  upon  which  the  analyses  of  the  second  administration  were  based 
varied  from  specialty  to  specialty  and  from  the  first  half  to  the  second  half  of  the  same  speci¬ 
alty.  The  number  of  booklets  ranged  from  a  low  of  66  for  the  first  half  of  the  Accounting  He 
Finance  specialty,  to  a  high  of  163  for  the  Ground  Radio  Operator  specialty.  The  unequal  num¬ 
bers  did  not  affect  the  analysis  of  the  data. 
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Variables  Used  in  Analyses  of  the  Task  Ratings 

The  variables  used  in  the  task  analyses  are  identified  in  Table  12.  Variable  X'j9,  the 
Task  Statement  P  Value,  represents  the  ratio  of  the  number  of  incumbents  who  wrote  in  a  task 
to  the  total  number  who  could  have  written  it  in.  In  the  case  of  omitted  tasks,  the  number  who 
could  have  written  it  in  was  equal  to  120,  the  number  of  incumbents  in  a  given  specialty  who 
took  a  particular  form  in  the  initial  administration.  In  the  case  of  new  statements,  the  denom¬ 
inator  was  360,  the  total  sample  for  the  AFSC  in  the  initial  administration.  The  second  vari¬ 
able,  X4q,  was  the  percentage  of  incumbents  in  an  AFSC  performing  a  task.  This  measure  was 
derived  from  the  data  obtained  with  t!  e  revised  inventory  in  the  second  administratio ! 

Variable  represents  the  score  derived  from  the  Time  ratings.  Incumbents  rated  tasks 
in  terms  of  the  average  time  necessary  to  do  a  task  once.  To  characterize  a  task  in  terms  of 
the  time  factor,  it  was  necessary  to  convert  the  responses  to  a  common  unit.  Ratings  on  the 
inventories  were  in  terms  of  seconds,  minutes,  and  hours.  For  purposes  of  analysis,  all  the 
time  data  were  converted  to  seconds. 

In  characterizing  a  task  on  the  Time  variable,  the  median  was  selected  as  the  appropriate 
measure  of  central  tendency.  The  reason  for  choosing  medians  is  apparent  from  an  example 
which  illustrates  why  the  distributions  of  ratings  were  sometimes  very  skewed.  Suppose  that 
a  Ground  Radio  Operator  is  rating  a  statement  like,  "I  have  handled  tactical  traffic.”  Most 
incumbents  will  react  to  each  transmission  of  tactical  traffic  as  a  separate  and  distinct  event. 
These  men  might  say  that  a  typical  transmission  of  tactical  traffic  takes  about  two  minutes. 

An  occasional  incumbent,  however,  will  respond  by  saying  "I  handle  tactiial  traffic  one  a 
day-all  day!  It  takes  me  8  hours  every  time  1  do  it.”  If  this  man  were  to  rate  a  statement  on 
the  time  dimension  with  a  value  of  8  hours  and  the  majority  of  the  people  rated  the  task  near 
2  minute'-  it  is  apparent  that  a  mean  of  the  time  ratings  would  be  a  poor  representation  of  the 
data. 

The  problems  with  respect  to  the  Frequency  ratings,  X42,  were  much  the  same  as  with 
average  Time  ratings.  Responses  had  to  be  converted  to  a  common  unit  and  a  measure  of 
central  tendency,  again  a  median,  was  computed  to  summarize  the  information  from  different 
incumbents.  All  ratings  were  reduced  to  number  of  times  per  year. 

Variable  X43  was  derived  from  the  Training  &  Experience  ratings.  Since  a  simple  4-point 
scale  was  used  for  this  factor,  a  mean  was  selected  as  the  measure  of  central  tendency  for 
summarizing  the  information  collected  from  the  incumbents. 

Obtaining  values  for  all  tasks  on  variables  X41,  X42,  and  X43,  was  itself  a  laborious 
job  which  would  not  have  been  feasible  without  the  use  of  a  computer.  The  routines  for  calcu¬ 
lating  means  and  medians  were  a  part  of  the  program  written  for  the  CDC  1604  computer.  In 
one  sense,  it  is  better  to  call  the  medians  which  were  calculated  "middle  numbers  since  the 
procedure  used  in  calculating  these  values  was  to  order  all  responses  and  select  the  middle 
value  when  an  odd  number  of  observations  were  available  and  to  average  the  two  middle  values 
when  an  even  number  of  observations  were  available  for  a  task.  The  number  of  observations 
on  which  medians  and  means  were  based  varied  according  to  the  number  who  reported  perform¬ 
ing  the  task. 

For  the  task  analyses,  only  new  or  omitted  statements  that  were  identically  worded  on 
the  revised  form  and  the  scoring  form  of  the  inventory  were  included.  The  number  of  statements 
used  for  each  specialty  was: 

Ground  Radio  Operator  215 

Automotive  Repairman  420 

Aircraft  Hydraulic  Repainnan  220 

Accounting  8c  Finance  Specialist  402 
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Table  12.  Variables  of  Task  Rating  Analyses 


Variable 

Name 

Variable 

Name 

*39 

Task  Statement  P  Value 

*4  5 

Accounting  &  Finance  Specialist 

*40 

Percent  of  AFSC  who  perform  task 

*46 

Automotive  Repairman 

*41 

Time,  in  seconds 

*47 

Aircraft  Hydraulic  Repairman 

*4  2 

Frequency  (times  per  year) 

*48 

Omitted  from  Form  80  or  90 

*4  3 
*44 

Training  &  Experience  Rating 

Ground  Radio  Operator 

*49 

New  Statement 

Table  13.  Multiple  Correlations  From  the  Task  Analyses 


(N  -  123 3) 


Variables 

Included 

Content  Included 

x39 

task  state¬ 
ment  P  value 

X^o 

percent  who 
perform  task 

*40  “  *47 

Item  Ratings,  AFSC 

.507 

*40  -  *49 

Item  Ratings,  AFSC,  New  vs 
Omitted  statements 

.524 

*44  “  *47 

AFSC 

.338 

.320 

*48  _  *49 

New  vs  Omitted  Statements 

.164 

.133 

*39-  *41“ *47 

P-Value,  Task  Ratings,  AFSC 

.575 

*39  *41“  *49 

Variables 

Deleted 

P-Value,  Task  Ratings,  AFSC, 

New  vs  Omitted  Statements 

Content  Deleted 

.575 

*40'  *48'  *49 

Percent  who  Perform  Task, 

New  vs  Omitted  Statements 

.342 

*41'  *48’  *49 

Time  Rating,  New  vs  Omitted 
Statements 

.507 

*42’  *48’  *49 

Frequency,  New  vs  Omitted 
Statements 

.516 

*44  “  ^9 

AFSC,  New  vs  Omitted  State¬ 
ments 

.440 

.505 

*39'  *41  “  *43'  *48’  *49 

P  Value,  Task  Ratings,  New 
vs  Omitted  Statements 

.320 
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Variables  X44,  X4J,  X46,  and  X47  represent  the  four  specialties  in  the  task  analyses. 
Variables  X4g  and  X49  were  used  to  identify  a  task  as  one  which  had  been  omitted  from  either 
Form  80  or  Form  90,  or  as  a  new  statement  obtained  from  the  write-ins  of  incumbents. 

Results  of  Task  Analyses 

Variables  X41  through  X49  in  the  task  analyses  were  used  as  predictor  variables.  Vari¬ 
ables  Xj9  and  X40  were  used  as  both  predictor  and  criterion  variables.  The  multiple  correla¬ 
tions  obtained  with  the  criterion  variables  are  presented  in  Table  13. 

The  highest  multiple  correlation  obtained  with  X39>  the  proportion  who  wrote  in  a  task, 
was  .524.  The  most  important  single  predictor  of  recoverability  (X}9)  was  the  percentage  of 
incumbents  who  perform  a  task  (X40),  with  a  correlation  of  .420.  The  larger  the  percentage 
of  incumbents  who  perform  a  task,  the  greater  the  likelihood  that  it  will  be  written  in.  The 
AFSC  variables  were  the  next  most  important  predictors,  correlating  as  a  set  .338  with  X}9. 
Time,  Frequency,  and  Training  &  Experience  variables  had  insignificant  correlations  with  x39. 
The  deletion  of  these  variables  from  the  regression  equation  had  negligible  effect  on  the  over¬ 
all  correlation. 

When  X40  was  used  as  the  crite-ion  and  all  the  other  variables  as  predictors,  a  multiple 
correlation  of  .575  was  obtained.  The  single  most  important  predictor  of  X40  was  XJ9,  the 
Task  Statement  P  Value  with  the  correlation  of  .420.  Correlations  with  other  predictors  are 
shown  in  Table  13. 

The  results  of  the  task  analyses  indicate  that  the  likelihood  that  a  statement  will  be 
written  in  is  relatively  independent  of  the  task-rating  factors.  Apparently  incumbents  remember 
tasks,  even  tasks  which  they  have  done  infrequently  or  which  take  every  little  time.  If  a  task 
inventory  were  a  free-recall  situation,  perhaps  incumbents  would  remember  first  those  tasks 
which  had  been  done  most  frequently  and  those  which,  on  the  average,  took  the  longest  amount 
of  time.  Eventually,  however,  even  less  frequently  performed  tasks  would  be  recalled.  Since 
the  task  inventory  is  an  aid  to  recall  with  a  high  degree  of  facilitation  of  memory  coming  from 
the  organization  of  the  inventory,  tasks  low  on  the  frequency  and  time  scales  seem  to  be  remem¬ 
bered  about  as  well  as  tasks  higher  on  these  scales. 

Incumbents  write  in  only  tasks  they  have  performed.  It  is,  therefore,  not  surprising  that 
the  percentage  of  incumbents  who  have  performed  a  task  is  a  fairly  good  predictor  of  the  pro¬ 
portion  of  incumbents  who  write  in  a  task. 


5.  DISCUSSION 

The  present  investigation  revealed  some  useful,  interesting,  and  unexpected  information 
about  the  task  inventory  method.  Perhaps  most  unexpected  was  the  finding  that  the  original 
inventories  produced  from  standard  sources  describing  the  specialties  were  so  incomplete. 
Three  forms  of  the  original  inventories  were  prepared,  principally  to  insure  that  some  of  the 
inventories  given  on  the  first  administration  would  be  incomplete.  As  it  turned  out,  the  number 
of  missing  statements  was  sufficiently  large  that  the  study  might  have  been  conducted  without 
length  of  form  being  used  as  a  variable.  The  inclusion  of  Form  did  emphasize  one  point.  The 
fact  that  sizable  samples  of  incumbents  were  necessary  to  recover  intentionally  omitted  state¬ 
ments  indicated  that  comparable  results  with  new  statements  have  some  generality.  If  omitted 
statements  had  been  recovered  with  relative  ease  with  small  samples  while  new  statements 
required  large  samples,  it  might  have  been  suggested  that  the  standards  for  accepting  incum¬ 
bents’  write-ins  as  statements  were  too  lax. 
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The  obtained  data  and  the  regression  equations  relating  the  yield  measures  to  sample 
size  were  also  somewhat  unexpected.  Over  the  range  of  sample  sizes  from  20  to  60,  the  regres- 
sion  equations  for  the  group  analyses  criteria,  X,  and  X2,  are  nearly  linear.  There  was  almost 
as  much  gain  in  yield  by  increasing  a  sample  from  size  40  to  size  60  as  there  was  from  increas¬ 
ing  a  sample  from  size  20  to  size  40.  For  samples  of  size  60,  neither  Xj  nor  X2  is  close  to 
its  upper  limit  of  1.00. 

The  reasons  for  these  findings  were  discussed  earlier  and  are  fairly  obvious  from  the 
data  of  Table  8.  Most  of  the  statements  which  were  written  in  by  incumbents  were  not  written 
in  very  many  times.  In  other  words,  the  majority  of  the  statements  have  low  probabilities  of 
recovery.  In  every  specialty,  over  50  percent  of  the  statements  were  written  no  more  than  four 
times.  In  the  Ground  Radio  Operator  specialty,  48  percent  of  the  statements  were  written  in 
only  once. 

Shat  is  the  answer  to  the  question  of  sample  size?  Is  it  possible  to  use  small  samples 
to  achieve  the  same  yield  of  information?  It  was  hoped  that  the  individual  analyses  would 
indicate  piedictor  variables  making  it  possible  to  preselect  incumbents  who  would  yield  large 
numbers  of  statements.  The  multiple  correlations  of  the  individual  analyses  were  too  low  to 
indicate  much  hope  for  the  particular  predictors  used.  The  present  results,  however,  should 
not  be  regarded  as  negative  with  respect  to  all  possible  predictors.  In  retrospect,  motivational 
variables  seem  a  likely  basis  for  selection.  As  shown  in  Table  7,  a  great  many  men  wrote  in 
no  acceptable  statements.  Many  men  wrote  in  nothing  at  all,  not  even  an  unacceptable  state¬ 
ment.  It  should  be  possible  to  eliminate  the  large  number  of  men  having  so  little  interest  in 
completing  the  inventory.  Part  of  this  lack  of  interest  was  due  to  the  newness  and  experimental 
nature  of  the  inventory  method.  An  educational  program  on  the  importance  of  the  job  information 
and  greater  familiarity  with  the  procedure  would  probably  reduce  the  number  of  men  who  write 
in  nothing.  Changes  in  the  method  of  administration,  such  as  allowing  the  incumbents  to  com¬ 
plete  the  inventories  in  their  work  areas  over  a  period  of  time,  might  increase  the  number  of 
write-ins  per  individual. 

Two  further  questions  might  be  asked  with  respect  to  the  adoption  of  a  procedure  for 
selecting  incumbents  who  are  potentially  most  useful  in  providing  information.  One  question 
concerns  the  yield  of  information  if  the  "best  possible  samples”  had  been  drawn  in  the  present 
research.  A  good  approximation  of  a  best  possible  sample  is  obtained  by  examining  the  re¬ 
sponses  of  the  individuals  who  provided  the  greatest  number  of  new  statements.  This  was 
done  for  samples  of  size  20  in  each  of  the  four  specialties.  The  20  individuals  who  wrote 
in  the  greatest  number  of  statements  in  each  specialty  were  selected,  and  a  value  of  X2  was 
calculated  for  each  specialty.  These  values  were: 


Ground  Radio  Operator  .624 

Automotive  Repairman  .614 

Aircraft  Hydraulic  Repairman  .742 

Accounting  &  Finance  Specialist  .621 


From  these  results  it  is  apparent  that  there  is  considerable  room  for  selection  procedure  to 
increase  yield  of  information.  These  "best”  yields  are  still  considerably  below  the  maximum 
yield  of  1.00,  reflecting  the  fact  that  many  statements  are  written  in  by  only  a  few  incumbents. 
The  second  question  is  whether  a  selection  procedure  would  result  in  biases  in  the  types  of 
information  recovered.  Such  an  outcome  is  always  a  possibility,  but  it  seems  unlikely  unless 
the  selection  procedure  resulted  in  an  unusual  distribution  of  incumbents  over  bases  or  geo¬ 
graphical  areas. 
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Selection  is  not  the  only  way  in  which  yields  might  be  increased.  In  this  study,  the 
incumbent  was  instructed  to  write  in  statements  describing  tasks  he  had  performed  Junng  the 
past  year.  This  instruction  contained  two  resttictions,  one  on  time,  and  the  second  on  experi¬ 
ence.  A  number  of  men  commented  that  they  had  performed  a  large  number  of  the  listed  tasks 
but  not  within  the  past  year.  Sometimes  men  were  in  school,  and  sometimes  a  man's  assign¬ 
ment  duting  the  past  year  had  restricted  his  activities  to  a  very  narrow  part  of  his  specialty. 
Othet  men  indicated  that  they  had  responsibilities  which  were  principally  supervisory,  and 
though  they  had  not  performed  them  themselves,  they  had  seen  the  tasks  performed  many  times 
during  the  past  year. 

It  would  seem  to  be  a  less  serious  error  to  fail  to  delete  an  outdated  task  than  to  fail  to 
include  a  new  task  in  a  specialty.  Therefore  thete  migh*  be  good  reason  for  permitting  men  to 
write  in  any  rnsk  which  they  know  to  have  been  performed  as  a  part  of  the  specialty  by  any  man. 
This  suggestion  seems  applicable  only  for  the  phase  in  which  the  inventory  is  being  constructed. 
It  would  result  in  the  inclusion  of  some  outdated  statements,  but  it  should  also  increase  the 
overall  level  of  yield.  The  problems  associated  with  this  procedure  need  to  be  investigated. 

Motivational  variables  have  already  been  suggested  as  possible  bases  for  selecting  incum¬ 
bents  who  will  provide  high  yields  of  information.  Motivational  techniques  might  be  used  to 
increase  the  yield  of  men  once  they  are  selected.  In  the  present  investigation  there  were  no 
attempts  to  indicate  to  the  men  who  completed  the  inventories  that  they  were  specially  selected 
for  their  experience  and  competence.  There  was  no  particular  pressure  to  do  a  good  job  in 
completing  the  inventory.  There  was  no  competition,  no  suggestion  that  the  incumbents  must 
write  in  at  least  some  statements,  and  no  indication  that  the  kind  of  effort  made  was  tied  to  a 
man’s  future  in  the  Air  Force.  There  are  certain  restrictions  on  the  types  ofincentives  that 
can  be  given  to  airmen,  but  any  move  in  this  direction  would  be  expected  to  improve  the  yield 
Of  information. 


6.  CONCLUSIONS 

Using  the  task  inventory  method,  a  great  deal  of  job  information  was  obtained  from  360 
incumbents  in  each  of  four  airman  specialties  which  was  not  available  in  source  materials 
such  as  specialty  descriptions,  OJT  programs,  job  training  standards,  and  APT  outlines.  The 
numbers  of  task  statements  extracted  from  the  source  materials  and  from  the  combination  of 


source  materials  and  incumbents’ 

Specialty 

write-ins  were: 

Source  Materials 

Source 

Materials  &  Write-ins 

Ground  Radio  Operator 

84 

270 

Accounting  &  Finance  Specialist 

177 

546 

Automotive  Repairman 

474 

750 

Aircraft  Hydraulic  Repairman 

345 

408 

The  task  statements  obtained  from  the  write-ins  of  the  incumbents  contained  significant 
aspects  of  the  specialties  which  had  not  been  previously  described  in  the  source  materials. 
Some  of  the  new  tasks  were  due  to  changes  in  the  nature  of  the  specialty  or  equipment  used 
(such  as  the  shift  to  EDP  equipment  and  methods  in  the  Accounting  &  Finance  specialty)  and 
other  task  statements’tepresented  important  aspects  of  the  specialty  which  had  escaped  de¬ 
tection  or  description  by  the  occupational  analysis  method  previously  used. 
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A  "umber  of  personal  characteristics  of  incumbents,  such  as  age,  rank,  proficiency  test 
score,  aptitude  indexes,  education,  and  length  of  service  were  investigated  to  determine  whether 
any  of  them  would  be  useful  in  selecting  individuals  who  would  give  either  a  significantly 
1  greater  quantity  or  quality  of  job  information.  None  of  the  characteristics  investigated,  either 

singly  or  in  combination,  were  related  sufficiently  to  quantity  or  quality  of  job  information 
yielded  to  justify  their  jise  for  this  purpose. 

The  relationships  between  several  measures  of  the  quantity  of  job  information  obtained 
from  a  group  and  the  group's  characteristics  were  investigated.  The  relationships  were  high, 

,  with  AFSC  and  sample  size  the  major  factors  in  the  relationships.  Regression  equations  ex¬ 

pressing  the  relationships  between  sample  size  and  the  yield  of  job  information  showed  that 
the  trend  of  the  relationship  was  linear,  over  the  range  of  sample  sizes  (20  to  60)  studied, 
indicating  that  the  yield  of  job  information  would  probably  continue  to  increase  with  increase 
in  sample  size  beyond  60.  This  result  was  related  to  the  low  probability  of  a  given  task  state¬ 
ment  being  written  in. 

The  relationship  between  the  likelihood  that  a  task  statement  would  be  written  in  by  job 
incumbents  and  certain  characteristics  of  a  task  (such  as  how  long  it  takes  to  perform  a  task, 
how  frequently  it  is  performed,  the  percentage  of  the  specialists  who  perform  the  task,  and  the 
amount  of  training  and  experience  needed)  were  investigated.  The  most  important  predictors 
were  the  percentage  of  incumbents  who  perform  a  task  and  the  AFSC.  Significant  differences 
in  the  likelihood  of  a  task  being  written  in  were  present  among  the  four  specialties  studied. 

The  factors  of  time  to  perform,  frequency,  and  amount  of  training  and  experience  needed  to 
perform  the  task  did  not  add  significantly  to  the  relationship  between  the  likelihood  of  a  task 
statement  being  written  in  and  the  two  most  important  factors  of  percentage  who  perform  a  task 
and  AFSC. 
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Appendix:  INSTRUCTIONS  FOR  RATING  TASKS 


INSTRUCTIONS  FOR  TIMF  RATINGS 


For  each  of  the  tasks  you  have  done  in  the  past  year-those  which  you  checked  "Yes” 
in  the  inventory  booklet-you  are  to  provide  information  on  the  amount  of  time  it  usually  takes 
to  perform  the  task  once. 

If  you  are  asked  how  much  time  it  usually  takes  to  perform  a  particular  task  once,  you 
might  answer  in  terms  of: 

seconds  (Sec), 
minutes  (Min), 
hours  (Mr). 

Choose  the  most  convenient  unit  for  each  task.  If,  on  the  average,  it  takes  30  minutes 
to  do  a  task  one  time,  next  to  the  number  for  that  task  you  would  write: 

30  MIN 

Use  the  accompanying  sheets  to  make  your  ratings. 


INSTRUCTIONS  FOR  FRFQUKNCY  RATINGS 

For  each  of  the  tasks  which  you  have  done  in  the  past  year-those  which  you  checked 
"Yes”  in  the  inventory  booklet -you  are  to  provide  information  on  how  frequently  you  have 
performed  the  task. 

If  you  are  asked  how  frequently  you  do  a  task,  how  would  you  answer?  -  You  might  say: 

a  certain  number  of  times  per  hour  (Hr), 
a  certain  number  of  times  per  day  (Day), 
a  certain  number  of  times  per  week  (Vi k), 
a  certain  number  of  times  per  month  (Mo),  or 
a  certain  number  of  times  per  year  (Yr). 

When  you  rate  the  task  statements  in  the  booklet,  for  each  statement  choose  the  unit 
which  seems  best  to  you.  Use  the  abbreviations  shown  above.  For  example,  if  you  want  to 
report  that  you  do  a  task  on  the  average  of  three  times  per  week,  next  to  the  number  for  that 
task  you  would  write: 

3  times  per  WK 

Use  the  accompanying  sheets  to  make  your  ratings. 
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INSTRUCTIONS  FOR  TRAINING  At  FXPi  RIKNCF  RATINGS 


Tasks  differ  in  the  amount  of  training  and/or  experience  needed  in  order  to  do  them 
proficiently.  Some  tasks  can  be  done  well  with  relatively  little  training  or  experience.  Other 
tasks  require  considerable  schooling  or  on-the-job  learning. 

For  each  of  the  tasks  you  have  done  in  the  past  year— those  which  you  checked  "Yes’" 
in  the  inventory  booklet— you  are  to  indicate  the  amount  of  training  and/or  experience  required 
in  order  to  do  the  task  proficiently.  Kach  task  is  to  be  rated  according  to  whether 

(a)  less  than  average, 

(b)  avenge, 

(c)  more  than  average,  or 

(d)  considerably  more  than  average 

training  and/or  experience  is  needed  to  perform  it  proficiently. 

In  rating  each  task,  think  about  it  in  relation  to  all  the  tasks  you  have  ever  done  as  a 
part  of  your  specialty. 

Make  your  ratings  on  the  accompanying  sheets  by  checking  (  X)  in  the  appropriate  space 
next  to  the  number  of  each  task  you  have  done  in  the  past  year. 
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